Oct 30 12:35:13.829882 ip-10-0-142-36 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 12:35:14.287115 ip-10-0-142-36 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:35:14.287115 ip-10-0-142-36 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 12:35:14.287115 ip-10-0-142-36 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:35:14.287115 ip-10-0-142-36 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:35:14.293237 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293151    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:14.293237 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293234    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:14.293237 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293239    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:14.293237 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293242    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293245    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293250    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293271    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293278    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293281    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293284    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293288    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293291    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293297    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293300    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293303    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293306    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293309    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293311    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293314    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293316    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293321    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293325    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:14.293403 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293328    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293331    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293337    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293340    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293342    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293345    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293348    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293351    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293355    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293357    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293360    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293363    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293365    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293368    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293371    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293376    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293379    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293381    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293384    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293387    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:14.293861 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293390    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293392    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293395    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293398    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293400    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293404    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293407    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293412    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293414    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293417    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293421    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293423    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293426    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293429    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293431    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293434    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293437    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293439    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293442    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293445    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:14.294382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293450    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293453    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293456    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293458    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293461    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293466    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293470    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293473    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293477    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293480    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293483    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293487    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293492    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293495    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293498    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293501    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293505    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293508    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293510    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:14.294857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293513    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293516    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293518    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293521    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293524    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293527    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293532    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293535    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293538    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293541    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293543    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293546    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293549    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293552    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293554    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293557    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293560    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293562    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293569    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293572    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:14.295303 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293575    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293577    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293580    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293582    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.293585    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294875    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294882    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294886    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294889    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294892    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294895    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294898    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294900    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294906    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294910    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294914    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294917    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294919    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294922    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294925    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:14.295770 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294928    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294931    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294934    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294936    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294939    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294942    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294947    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294949    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294952    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294955    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294957    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294960    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294963    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294965    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294968    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294971    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294973    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294976    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294979    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294984    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:14.296245 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294987    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294990    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294993    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294996    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.294998    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295001    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295004    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295007    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295010    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295012    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295015    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295018    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295023    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295025    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295028    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295031    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295033    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295036    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295038    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:14.296726 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295041    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295044    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295046    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295049    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295052    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295057    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295059    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295062    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295065    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295067    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295070    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295073    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295076    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295079    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295082    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295084    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295088    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295093    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295096    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:14.297185 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295101    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295105    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295108    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295111    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295114    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295117    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295119    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295122    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295125    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295127    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295133    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295135    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295138    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295141    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295143    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295146    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295149    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295152    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295155    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:14.297649 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295157    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295160    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295163    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295166    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295171    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295175    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295179    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295183    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295187    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295192    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295195    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295198    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295201    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.295204    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296023    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296029    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296035    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296038    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296041    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:14.298126 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296046    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296050    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296053    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296057    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296059    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296063    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296066    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296069    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296071    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296077    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296081    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296083    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296087    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296089    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296092    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296095    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296097    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296100    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296103    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:14.298570 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296106    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296109    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296111    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296117    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296119    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296122    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296125    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296128    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296131    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296134    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296138    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296141    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296144    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296147    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296150    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296155    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296157    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296160    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296163    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296165    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:14.299058 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296168    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296171    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296174    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296177    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296179    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296182    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296185    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296188    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296193    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296196    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296199    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296202    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296204    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296207    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296210    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296213    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296215    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296218    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296221    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296223    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:14.299557 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296234    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296237    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296239    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296242    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296244    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296247    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296250    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296253    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296256    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296258    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296263    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296266    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296269    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296274    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296277    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296280    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296282    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296285    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296288    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:14.300038 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296291    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296293    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296297    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296300    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296303    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296308    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296311    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296313    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296316    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296319    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296322    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296324    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296327    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296330    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296332    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296335    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296337    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296340    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296345    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296348    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:14.300481 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296351    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296353    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.296356    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296753    2581 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296779    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296788    2581 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296793    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296798    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296801    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296806    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296811    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296814    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296817    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296821    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296824    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296828    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296832    2581 flags.go:64] FLAG: --cgroup-root=""
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296835    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296838    2581 flags.go:64] FLAG: --client-ca-file=""
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296841    2581 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296844    2581 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296851    2581 flags.go:64] FLAG: --cluster-domain=""
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296854    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 12:35:14.301000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296857    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296861    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296864    2581 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296877    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296882    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296885    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296890    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296893    2581 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296898    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296901    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296905    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296908    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296913    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296919    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296923    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296927    2581 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296930    2581 flags.go:64] FLAG: --enable-server="true"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296934    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296942    2581 flags.go:64] FLAG: --event-burst="100"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296946    2581 flags.go:64] FLAG: --event-qps="50"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296951    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296955    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296960    2581 flags.go:64] FLAG: --eviction-hard=""
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296966    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 12:35:14.301532 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296970    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296975    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296978    2581 flags.go:64] FLAG: --eviction-soft=""
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296983    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296987    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296992    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296996    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.296998    2581 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297003    2581 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297007    2581 flags.go:64] FLAG: --feature-gates=""
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297013    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297016    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297019    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297023    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297027    2581 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297030    2581 flags.go:64] FLAG: --help="false"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297033    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297038    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297041    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297044    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297049    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297053    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297056    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 12:35:14.302133 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297059    2581 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297061    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297065    2581 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297067    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297071    2581 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297074    2581 flags.go:64] FLAG: --kube-reserved=""
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297077    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297080    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297083    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297086    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297089    2581 flags.go:64] FLAG: --lock-file=""
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297092    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297095    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297099    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297110    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297113    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297117    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297120    2581 flags.go:64] FLAG: --logging-format="text"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297123    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297126    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297129    2581 flags.go:64] FLAG: --manifest-url=""
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297132    2581 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297138    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297141    2581 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297145    2581 flags.go:64] FLAG: --max-pods="110"
Oct 30 12:35:14.302703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297148    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297152    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297155    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297159    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297162    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297169    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297172    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297181    2581 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297184    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297187    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297191    2581 flags.go:64] FLAG: --pod-cidr=""
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297194    2581 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297197    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297200    2581 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297203    2581 flags.go:64] FLAG: --port="10250"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297206    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297210    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00a767a3fd94df4b3"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297213    2581 flags.go:64] FLAG: --qos-reserved=""
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297216    2581 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297219    2581 flags.go:64] FLAG: --register-node="true"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297222    2581 flags.go:64] FLAG: --register-with-taints=""
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297226    2581 flags.go:64] FLAG: --registry-burst="10"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297229    2581 flags.go:64] FLAG: --registry-qps="5"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297232    2581 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297235    2581 flags.go:64] FLAG: --reserved-memory=""
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297239    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 12:35:14.303333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297242    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297245    2581 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297248    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297252    2581 flags.go:64] FLAG: --runonce="false"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297255    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297258    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297261    2581 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297264    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297267    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297271    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297274    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297278    2581 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297281    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297284    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297287    2581 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297290    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297293    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297296    2581 flags.go:64] FLAG: --system-cgroups=""
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297299    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297305    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297308    2581 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297311    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297316    2581 flags.go:64] FLAG: --tls-min-version=""
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297319    2581 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297322    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 12:35:14.303990 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297324    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297328    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297332    2581 flags.go:64] FLAG: --v="2"
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297337    2581 flags.go:64] FLAG: --version="false"
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297341    2581 flags.go:64] FLAG: --vmodule=""
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297345    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297348    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297498    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297502    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297506    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297509    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297511    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297515    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297518    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297520    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297523    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297525    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297529    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297532    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297535    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297538    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:14.304631 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297540    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297543    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297546    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297549    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297551    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297554    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297557    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297560    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297562    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297565    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297567    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297570    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297573    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297577    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297580    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297582    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297585    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297587    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297590    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297592    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:14.305142 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297609    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297612    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297614    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297617    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297619    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297622    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297625    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297627    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297630    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297633    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297637    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297641    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297644    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297647    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297650    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297652    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297655    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297658    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297661    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:14.305608 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297663    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297666    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297669    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297671    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297674    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297676    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297681    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297683    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297686    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297689    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297691    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297694    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297697    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297699    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297702    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297704    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297707    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297710    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297712    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297715    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:14.306074 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297717    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297720    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297722    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297726    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297728    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297731    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297733    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297736    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297738    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297741    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297743    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297745    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297748    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297750    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297753    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297755    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297758    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297760    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297764    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:14.306553 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297767    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297769    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297772    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297775    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297777    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297782    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297784    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297787    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297790    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297794    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297797    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297800    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297803    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297805    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.297834    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297869    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:14.307016 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297874    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297877    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297880    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297883    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297885    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297888    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297890    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297893    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297895    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297898    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297901    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297904    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297906    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297909    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297912    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297915    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297918    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297921    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297923    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:14.307397 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297927    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297930    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297934    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297938    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297941    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297944    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297947    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297949    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297952    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297954    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297957    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297960    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297962    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297965    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297967    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297970    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297974    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297976    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297979    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:14.307857 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297982    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297984    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297987    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297990    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297993    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297995    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.297998    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298000    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298004    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298008    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298010    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298013    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298015    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298018    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298020    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298023    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298027    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298030    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298032    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298035    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:14.308294 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298037    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298040    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298042    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298045    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298048    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298050    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298053    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298055    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298058    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298062    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298065    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298069    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298072    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298075    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298078    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298081    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298083    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298086    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298088    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:14.308899 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298091    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298094    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298098    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298100    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298103    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298106    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298109    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298111    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298114    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298118    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298121    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298124    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298126    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298129    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298132    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298135    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298138    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298140    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298143    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298145    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:14.309362 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298148    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298150    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298153    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298157    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298160    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298163    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298166    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.298168    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.305352    2581 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.305367    2581 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.305380    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.305385    2581 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305441    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305446    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305450    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305453    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305455    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305458    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305461    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:14.309903 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305464    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305466    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305469    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305472    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305475    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305477    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305480    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305482    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305485    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305488    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305491    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305494    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305497    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305499    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305502    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305505    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305512    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305515    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305518    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305520    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:14.310360 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305523    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305526    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305529    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305532    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305535    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305538    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305541    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305543    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305546    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305549    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305551    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305554    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305557    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305559    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305562    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305564    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305567    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305570    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305572    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305575    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:14.310850 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305578    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305580    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305583    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305585    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305588    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305590    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305593    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305609    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305612    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305618    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305622    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305625    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305628    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305631    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305635    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305638    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305641    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305643    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:14.311316 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305646    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305649    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305652    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305654    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305657    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305660    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305662    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305665    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305667    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305670    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305673    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305675    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305678    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305681    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305683    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305686    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305688    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305691    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305694    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305697    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:14.311753 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305699    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305702    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305704    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305707    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305710    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305713    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305716    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305718    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305722    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305725    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305727    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305730    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305733    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305735    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305738    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305740    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305743    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305745    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305748    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:14.312205 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305751    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:14.312749 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305754    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:14.312749 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.305785    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:35:14.312749 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305926    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:14.312749 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305931    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:14.312749 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305935    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:14.312749 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305937    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:14.312749 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305941    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:14.312749 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305943    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:14.312749 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305946    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:14.312749 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305948    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:14.312749 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305951    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:14.312749 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305954    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:14.312749 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305956    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:14.312749 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305959    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:14.312749 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305962    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305964    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305967    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305970    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305973    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305976    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305978    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305982    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305985    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305987    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305990    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305993    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305995    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.305998    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306001    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306003    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306006    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306009    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306011    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:14.313102 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306014    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306017    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306020    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306022    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306025    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306027    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306030    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306032    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306035    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306039    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306043    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306046    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306049    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306052    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306055    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306057    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306060    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306064    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306067    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:14.313545 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306070    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306072    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306075    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306078    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306081    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306084    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306086    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306089    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306091    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306094    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306097    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306099    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306102    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306104    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306107    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306110    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306112    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306115    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306118    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306120    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:14.314024 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306124    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306128    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306131    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306134    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306136    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306139    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306141    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306144    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306146    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306149    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306152    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306155    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306158    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306160    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306163    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306166    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306168    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306171    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306173    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:14.314587 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306176    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306179    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306181    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306184    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306186    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306189    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306191    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306194    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306196    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306199    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306201    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306204    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306206    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306209    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306211    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306214    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:14.315149 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306217    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.306239    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306271    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306275    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306279    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306281    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306285    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306289    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306293    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306296    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306300    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306303    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306306    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306308    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306311    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306313    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:14.315531 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306316    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306319    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306321    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306324    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306326    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306329    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306331    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306334    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306336    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306339    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306342    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306344    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306347    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306350    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306352    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306355    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306358    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306361    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306363    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306366    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:14.315937 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306368    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306372    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306376    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306378    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306381    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306384    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306387    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306389    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306392    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306394    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306397    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306400    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306403    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306405    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306408    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306410    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306413    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306415    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306418    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:14.316417 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306420    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306423    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306425    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306428    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306430    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306433    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306435    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306438    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306440    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306443    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306446    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306448    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306451    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306453    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306456    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306458    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306461    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306464    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306466    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306469    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:14.316912 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306472    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306474    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306477    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306479    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306482    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306485    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306487    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306490    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306492    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306495    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306497    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306500    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306502    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306505    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306508    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306510    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306512    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306515    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306517    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:14.317382 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306520    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306523    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306525    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306528    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306531    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306533    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306536    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306538    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306541    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306543    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306546    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306548    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306553    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:14.306556    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.306618    2581 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.309072    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.310148    2581 server.go:1014] "Starting client certificate rotation"
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.310243    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 12:35:14.317865 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.310270    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 12:35:14.318270 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.312543    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 12:35:14.318270 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.314032    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 12:35:14.325030 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.325017    2581 log.go:25] "Validated CRI v1 runtime API"
Oct 30 12:35:14.329164 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.329150    2581 log.go:25] "Validated CRI v1 image API"
Oct 30 12:35:14.330056 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.330027    2581 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 12:35:14.336215 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.336190    2581 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 f05e0c01-eaf2-4645-89d4-33cbcd4b548c:/dev/nvme0n1p4 fff9e7d8-1832-4521-9d95-6334c72f9fc4:/dev/nvme0n1p3]
Oct 30 12:35:14.336300 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.336214    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 12:35:14.343021 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.342908    2581 manager.go:217] Machine: {Timestamp:2026-10-30 12:35:14.341539088 +0000 UTC m=+0.381242769 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099212 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25e17718a8a609f0a32a2d110d1554 SystemUUID:ec25e177-18a8-a609-f0a3-2a2d110d1554 BootID:dfcad087-3a8d-4dd6-b5fe-b0735943db67 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e5:4d:34:e3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e5:4d:34:e3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:69:28:c3:7a:46 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 12:35:14.343021 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.343016    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 12:35:14.343185 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.343114    2581 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 12:35:14.346750 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.346721    2581 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 12:35:14.346901 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.346752    2581 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-36.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 12:35:14.346987 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.346905    2581 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 12:35:14.346987 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.346918    2581 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 12:35:14.346987 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.346953    2581 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 12:35:14.347867 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.347854    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 12:35:14.349141 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.349128    2581 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 12:35:14.354228 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.354214    2581 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 12:35:14.354364 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.354354    2581 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 12:35:14.355176 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.355167    2581 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 12:35:14.355221 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.355182    2581 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 12:35:14.355221 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.355193    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 12:35:14.355221 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.355201    2581 kubelet.go:405] "Adding apiserver pod source"
Oct 30 12:35:14.355221 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.355209    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 12:35:14.355536 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.355525    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 12:35:14.358054 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.358039    2581 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 12:35:14.364619 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.364595    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 12:35:14.365583 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.365571    2581 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 12:35:14.365629 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.365612    2581 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 12:35:14.365759 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.365748    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 12:35:14.365787 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.365764    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 12:35:14.365787 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.365770    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 12:35:14.365787 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.365776    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 12:35:14.365787 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.365784    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 12:35:14.365890 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.365790    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 12:35:14.366467 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.366449    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 12:35:14.366521 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.366468    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 12:35:14.366521 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.366476    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 12:35:14.366521 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.366482    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 12:35:14.366521 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.366497    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 12:35:14.366521 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.366506    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 12:35:14.367385 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.367375    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 12:35:14.367385 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.367385    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 12:35:14.371151 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.371137    2581 server.go:1263] "Started kubelet"
Oct 30 12:35:14.371247 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.371214    2581 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 12:35:14.371333 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.371254    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 12:35:14.371383 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.371358    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 12:35:14.371925 ip-10-0-142-36 systemd[1]: Started Kubernetes Kubelet.
Oct 30 12:35:14.372796 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.372761    2581 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 12:35:14.373224 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.373198    2581 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 12:35:14.379112 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.379083    2581 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 12:35:14.380900 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.380882    2581 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 12:35:14.380980 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.380904    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 12:35:14.381417 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.381081    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:14.381652 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.381630    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 12:35:14.381736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.381659    2581 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 12:35:14.381736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.381671    2581 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 12:35:14.382692 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.382493    2581 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 12:35:14.382784 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.382699    2581 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 12:35:14.382784 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.382462    2581 factory.go:153] Registering CRI-O factory
Oct 30 12:35:14.382784 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.382740    2581 factory.go:223] Registration of the crio container factory successfully
Oct 30 12:35:14.382919 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.382802    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 12:35:14.382919 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.382812    2581 factory.go:55] Registering systemd factory
Oct 30 12:35:14.382919 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.382820    2581 factory.go:223] Registration of the systemd container factory successfully
Oct 30 12:35:14.382919 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.382837    2581 factory.go:103] Registering Raw factory
Oct 30 12:35:14.382919 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.382850    2581 manager.go:1199] Started watching for new ooms in manager
Oct 30 12:35:14.384856 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.384836    2581 manager.go:319] Starting recovery of all containers
Oct 30 12:35:14.395738 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.395718    2581 manager.go:324] Recovery completed
Oct 30 12:35:14.399908 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.399889    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:35:14.402047 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.401827    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:35:14.402261 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.402252    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:35:14.402864 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.402848    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:35:14.404268 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.404254    2581 cpu_manager.go:226] "Starting" policy="none"
Oct 30 12:35:14.404268 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.404266    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:35:14.404371 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.404266    2581 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 12:35:14.404371 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.404290    2581 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 12:35:14.405511 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.405491    2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 12:35:14.405831 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.405817    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:35:14.405880 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.405844    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:35:14.405880 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.405857    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:35:14.406550 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.406538    2581 policy_none.go:50] "Start"
Oct 30 12:35:14.406592 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.406554    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 12:35:14.406592 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.406567    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 12:35:14.407178 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.407158    2581 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-36.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 12:35:14.409070 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.409059    2581 policy_none.go:44] "Start"
Oct 30 12:35:14.418324 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.418215    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-36.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 12:35:14.419445 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.418245    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-36.ec2.internal.18e34ecbec2a5684  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-36.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-36.ec2.internal,},FirstTimestamp:2026-10-30 12:35:14.371114628 +0000 UTC m=+0.410818310,LastTimestamp:2026-10-30 12:35:14.371114628 +0000 UTC m=+0.410818310,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-36.ec2.internal,}"
Oct 30 12:35:14.420751 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.420727    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 12:35:14.420823 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.420729    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 12:35:14.425098 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.425079    2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s7kbl"
Oct 30 12:35:14.432166 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.432147    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s7kbl"
Oct 30 12:35:14.433167 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.433137    2581 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 12:35:14.435473 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.435457    2581 manager.go:341] "Starting Device Plugin manager"
Oct 30 12:35:14.435565 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.435502    2581 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 12:35:14.435565 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.435514    2581 server.go:87] "Starting device plugin registration server"
Oct 30 12:35:14.435765 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.435753    2581 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 12:35:14.435801 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.435768    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 12:35:14.435933 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.435914    2581 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 12:35:14.436013 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.435989    2581 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 12:35:14.436013 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.435997    2581 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 12:35:14.436727 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.436553    2581 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 12:35:14.436795 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.436767    2581 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:14.449646 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.449562    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-36.ec2.internal.18e34ecbee0253fc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-36.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-36.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-36.ec2.internal,},FirstTimestamp:2026-10-30 12:35:14.402046972 +0000 UTC m=+0.441750654,LastTimestamp:2026-10-30 12:35:14.402046972 +0000 UTC m=+0.441750654,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-36.ec2.internal,}"
Oct 30 12:35:14.449741 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.449651    2581 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-36.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 12:35:14.512239 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.512216    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 12:35:14.514325 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.513352    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 12:35:14.514325 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.513370    2581 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 12:35:14.514325 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.513389    2581 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 12:35:14.514325 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.513420    2581 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 12:35:14.529062 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.529045    2581 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:35:14.536136 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.536123    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:35:14.536960 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.536944    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:35:14.537022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.536980    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:35:14.537022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.536998    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:35:14.537022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.537022    2581 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.555398 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.555354    2581 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.555398 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.555379    2581 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-36.ec2.internal\": node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:14.594088 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.594066    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:14.614380 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.614357    2581 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal"]
Oct 30 12:35:14.614441 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.614422    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:35:14.617167 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.617152    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:35:14.617216 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.617179    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:35:14.617216 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.617188    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:35:14.619462 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.619451    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:35:14.619639 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.619625    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.619678 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.619656    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:35:14.620278 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.620261    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:35:14.620349 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.620289    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:35:14.620349 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.620299    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:35:14.620349 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.620287    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:35:14.620349 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.620339    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:35:14.620529 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.620356    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:35:14.622574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.622559    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.622674 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.622587    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:35:14.623377 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.623361    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:35:14.623474 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.623385    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:35:14.623474 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.623399    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:35:14.641051 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.641030    2581 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-36.ec2.internal\" not found" node="ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.644031 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.644016    2581 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-36.ec2.internal\" not found" node="ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.683793 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.683770    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45b9204b780ab5797493be0f231a6750-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal\" (UID: \"45b9204b780ab5797493be0f231a6750\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.683875 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.683860    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45b9204b780ab5797493be0f231a6750-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal\" (UID: \"45b9204b780ab5797493be0f231a6750\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.683919 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.683883    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6766e4bd763f65c3c761bb65d5e5c301-config\") pod \"kube-apiserver-proxy-ip-10-0-142-36.ec2.internal\" (UID: \"6766e4bd763f65c3c761bb65d5e5c301\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.694871 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.694850    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:14.784673 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.784651    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45b9204b780ab5797493be0f231a6750-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal\" (UID: \"45b9204b780ab5797493be0f231a6750\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.784772 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.784675    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45b9204b780ab5797493be0f231a6750-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal\" (UID: \"45b9204b780ab5797493be0f231a6750\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.784772 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.784694    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6766e4bd763f65c3c761bb65d5e5c301-config\") pod \"kube-apiserver-proxy-ip-10-0-142-36.ec2.internal\" (UID: \"6766e4bd763f65c3c761bb65d5e5c301\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.784772 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.784750    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6766e4bd763f65c3c761bb65d5e5c301-config\") pod \"kube-apiserver-proxy-ip-10-0-142-36.ec2.internal\" (UID: \"6766e4bd763f65c3c761bb65d5e5c301\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.784772 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.784756    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45b9204b780ab5797493be0f231a6750-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal\" (UID: \"45b9204b780ab5797493be0f231a6750\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.784905 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.784761    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45b9204b780ab5797493be0f231a6750-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal\" (UID: \"45b9204b780ab5797493be0f231a6750\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.795797 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.795780    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:14.896110 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.896048    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:14.945001 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.944983    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.947132 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:14.947115    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:14.997141 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:14.997117    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:15.098191 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:15.098155    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:15.198617 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:15.198540    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:15.299372 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:15.299332    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:15.310283 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.310258    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 12:35:15.310437 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.310409    2581 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 12:35:15.400281 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:15.400247    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:15.428099 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.428076    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:35:15.429050 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.429031    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:35:15.429146 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.429068    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:35:15.429146 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.429084    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:35:15.438132 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.438100    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 12:30:15 +0000 UTC" deadline="2028-04-01 07:57:54.251703638 +0000 UTC"
Oct 30 12:35:15.438132 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.438127    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12451h22m38.813579625s"
Oct 30 12:35:15.500705 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:15.500649    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:15.601140 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:15.601109    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:15.660198 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:15.660154    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6766e4bd763f65c3c761bb65d5e5c301.slice/crio-c2ce37edc542d23f0715646c0c806e458b1cf6afdb37838f35b00e122f08fb4a WatchSource:0}: Error finding container c2ce37edc542d23f0715646c0c806e458b1cf6afdb37838f35b00e122f08fb4a: Status 404 returned error can't find the container with id c2ce37edc542d23f0715646c0c806e458b1cf6afdb37838f35b00e122f08fb4a
Oct 30 12:35:15.663482 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.663461    2581 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:35:15.665951 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.665931    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 12:35:15.671006 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:15.670984    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45b9204b780ab5797493be0f231a6750.slice/crio-fb8a16464c99266ad50f6c092a3fcb92ed6c2701b65333a20f9d5a63b1271d33 WatchSource:0}: Error finding container fb8a16464c99266ad50f6c092a3fcb92ed6c2701b65333a20f9d5a63b1271d33: Status 404 returned error can't find the container with id fb8a16464c99266ad50f6c092a3fcb92ed6c2701b65333a20f9d5a63b1271d33
Oct 30 12:35:15.701239 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:15.701214    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:15.701382 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.701364    2581 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:35:15.783344 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.783283    2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:15.806485 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.806463    2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:35:15.807965 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.807952    2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:15.830678 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.830656    2581 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:35:15.843125 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:15.843109    2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:35:16.356582 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.356555    2581 apiserver.go:52] "Watching apiserver"
Oct 30 12:35:16.357704 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.357675    2581 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:35:16.382663 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.382635    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 12:35:16.398363 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.398336    2581 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 12:35:16.398792 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.398763    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-kqrtg","kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal","openshift-cluster-node-tuning-operator/tuned-5frn2","openshift-network-operator/iptables-alerter-k9987","kube-system/global-pull-secret-syncer-ntzbv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal","openshift-multus/multus-additional-cni-plugins-cpq89","openshift-multus/network-metrics-daemon-xdcfv","openshift-network-diagnostics/network-check-target-29mpd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh","openshift-dns/node-resolver-cwcj7","openshift-ovn-kubernetes/ovnkube-node-jk5qv","kube-system/konnectivity-agent-p2z49","openshift-image-registry/node-ca-pt7kp"]
Oct 30 12:35:16.401506 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.401488    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.403822 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.403791    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:16.406150 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.406130    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.408403 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.408382    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:16.412153 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.412132    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:16.412245 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:16.412219    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:16.415505 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.415485    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:16.416883 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.416864    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 12:35:16.417010 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.416985    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 12:35:16.417221 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.417165    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 12:35:16.417332 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.417317    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 12:35:16.417401 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.417388    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 12:35:16.417526 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.417512    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gmdlh\""
Oct 30 12:35:16.418151 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.418128    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:16.418232 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:16.418195    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:16.418371 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.418351    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 12:35:16.418778 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.418748    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 12:35:16.418882 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.418836    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-6qrlt\""
Oct 30 12:35:16.421901 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.421876    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-x6m2k\""
Oct 30 12:35:16.422001 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.421978    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:16.422071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.422002    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 12:35:16.423035 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.422842    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.423035 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.422963    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 12:35:16.425249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.425233    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:16.425327 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:16.425285    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:16.427808 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.427784    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rkrpr\""
Oct 30 12:35:16.429310 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.429287    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.431999 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.431981    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:16.434650 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.434603    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.438814 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.438790    2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 12:35:16.438910 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.438872    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nfjqq"
Oct 30 12:35:16.482824 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.482803    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 12:35:16.494099 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494039    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/15bb3274-f845-4851-90ae-f4ab08e685c8-kubelet-config\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:16.494207 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494115    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/15bb3274-f845-4851-90ae-f4ab08e685c8-dbus\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:16.494207 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494146    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f86qq\" (UniqueName: \"kubernetes.io/projected/3ab05e61-6a22-4ced-abec-a528a15f9ba0-kube-api-access-f86qq\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.494207 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494175    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-sys-fs\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.494207 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494202    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-log-socket\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.494470 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494228    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-run-ovn-kubernetes\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.494470 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494252    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.494470 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494268    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-device-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.494470 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494284    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-kubelet\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.494470 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494299    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppcp2\" (UniqueName: \"kubernetes.io/projected/18fbde11-0050-4b55-bc23-a4e5e76ddc81-kube-api-access-ppcp2\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.494470 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494314    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d31facfa-368e-4417-b9ce-30c3a495f650-tmp\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.494470 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494330    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1263e83a-e0fd-4eb4-8141-5026aec6a631-iptables-alerter-script\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:16.494470 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494356    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-k8s-cni-cncf-io\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.494470 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494372    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-systemd\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.494470 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494386    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-systemd\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.494470 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494412    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce-agent-certs\") pod \"konnectivity-agent-p2z49\" (UID: \"7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce\") " pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:16.495034 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-registration-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.495034 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494510    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovn-node-metrics-cert\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.495034 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494535    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-lib-modules\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.495034 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494562    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-cni-multus\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.495034 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494578    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-etc-selinux\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.495034 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494593    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bm4p\" (UniqueName: \"kubernetes.io/projected/d31facfa-368e-4417-b9ce-30c3a495f650-kube-api-access-8bm4p\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.495034 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494614    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1263e83a-e0fd-4eb4-8141-5026aec6a631-host-slash\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:16.495034 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494807    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-cnibin\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.495034 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494860    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ab05e61-6a22-4ced-abec-a528a15f9ba0-cni-binary-copy\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.495034 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494917    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-socket-dir-parent\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.495034 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.494973    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:16.495034 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495017    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-hosts-file\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495045    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-env-overrides\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495078    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysconfig\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495118    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a594de5b-9748-4355-86b8-95ca7dc9dda1-serviceca\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495159    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lshc4\" (UniqueName: \"kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4\") pod \"network-check-target-29mpd\" (UID: \"a7e45249-4b17-415f-966d-af5321f182d5\") " pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495207    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-kubelet\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495235    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-run-netns\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495262    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovnkube-config\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495290    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-system-cni-dir\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495344    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-hostroot\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495370    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-conf-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495398    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6wnt\" (UniqueName: \"kubernetes.io/projected/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-kube-api-access-q6wnt\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495424    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-ovn\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495450    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-node-log\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495478    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovnkube-script-lib\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495501    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-modprobe-d\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.495606 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495516    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysctl-d\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495592    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysctl-conf\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495619    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-system-cni-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495659    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495695    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhm96\" (UniqueName: \"kubernetes.io/projected/4b56b071-d626-4d8d-bba6-f5f69fa95db0-kube-api-access-vhm96\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495721    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d31facfa-368e-4417-b9ce-30c3a495f650-etc-tuned\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495788    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-825sw\" (UniqueName: \"kubernetes.io/projected/1263e83a-e0fd-4eb4-8141-5026aec6a631-kube-api-access-825sw\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495833    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a594de5b-9748-4355-86b8-95ca7dc9dda1-host\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495861    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkj5v\" (UniqueName: \"kubernetes.io/projected/a594de5b-9748-4355-86b8-95ca7dc9dda1-kube-api-access-bkj5v\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495895    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-cni-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495916    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-netns\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495934    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-slash\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495971    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.495988    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-cni-netd\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496013    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-host\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496041    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-multus-certs\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.496286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496069    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-etc-kubernetes\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496135    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8xqw\" (UniqueName: \"kubernetes.io/projected/2758dbad-1f8a-4503-9dc2-6181518daeaf-kube-api-access-q8xqw\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496173    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-etc-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496236    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cni-binary-copy\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496289    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496333    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496391    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-sys\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496431    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-daemon-config\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496451    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-var-lib-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496489    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-run\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496526    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-var-lib-kubelet\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496556    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-socket-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496659    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cnibin\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496697    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496715    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-cni-bin\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.497051 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496752    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmn28\" (UniqueName: \"kubernetes.io/projected/e937e696-f801-42e7-9622-d774cca50028-kube-api-access-pmn28\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.497865 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496787    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-tmp-dir\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:16.497865 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496824    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-kubernetes\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.497865 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496857    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce-konnectivity-ca\") pod \"konnectivity-agent-p2z49\" (UID: \"7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce\") " pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:16.497865 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496888    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-systemd-units\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.497865 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.496945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-cni-bin\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.519462 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.519391    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal" event={"ID":"45b9204b780ab5797493be0f231a6750","Type":"ContainerStarted","Data":"fb8a16464c99266ad50f6c092a3fcb92ed6c2701b65333a20f9d5a63b1271d33"}
Oct 30 12:35:16.521700 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.521671    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal" event={"ID":"6766e4bd763f65c3c761bb65d5e5c301","Type":"ContainerStarted","Data":"c2ce37edc542d23f0715646c0c806e458b1cf6afdb37838f35b00e122f08fb4a"}
Oct 30 12:35:16.526880 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.526856    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 12:35:16.526986 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.526882    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 12:35:16.526986 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.526918    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 12:35:16.526986 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.526937    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 12:35:16.526986 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.526961    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 12:35:16.526986 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.526974    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 12:35:16.527256 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527155    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:16.527256 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527201    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 12:35:16.527256 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527211    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2vwzh\""
Oct 30 12:35:16.527256 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527247    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 12:35:16.527446 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527272    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 12:35:16.527446 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527276    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qjz2q\""
Oct 30 12:35:16.527446 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527289    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 12:35:16.527446 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527312    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 12:35:16.527446 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527328    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 12:35:16.527446 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527348    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-562t6\""
Oct 30 12:35:16.527446 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527350    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 12:35:16.527446 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527370    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 12:35:16.527446 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527418    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 12:35:16.527446 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527431    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 12:35:16.527446 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527422    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-scxms\""
Oct 30 12:35:16.527841 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.527470    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kkflg\""
Oct 30 12:35:16.598110 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598052    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-log-socket\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.598282 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598118    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-run-ovn-kubernetes\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.598282 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598140    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.598282 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598168    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-device-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.598282 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598181    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-log-socket\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.598282 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598186    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-kubelet\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.598282 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598244    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppcp2\" (UniqueName: \"kubernetes.io/projected/18fbde11-0050-4b55-bc23-a4e5e76ddc81-kube-api-access-ppcp2\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.598282 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598251    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.598282 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598272    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d31facfa-368e-4417-b9ce-30c3a495f650-tmp\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598297    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-run-ovn-kubernetes\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598304    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1263e83a-e0fd-4eb4-8141-5026aec6a631-iptables-alerter-script\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598333    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-k8s-cni-cncf-io\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598254    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-device-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598358    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-systemd\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598387    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-systemd\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598405    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-k8s-cni-cncf-io\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598412    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce-agent-certs\") pod \"konnectivity-agent-p2z49\" (UID: \"7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce\") " pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598440    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-registration-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598450    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-systemd\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovn-node-metrics-cert\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598489    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-lib-modules\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598514    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-cni-multus\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598542    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-etc-selinux\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598594    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-cni-multus\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598605    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-registration-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.598734 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598606    2581 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598735    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-etc-selinux\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598252    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-kubelet\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598783    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bm4p\" (UniqueName: \"kubernetes.io/projected/d31facfa-368e-4417-b9ce-30c3a495f650-kube-api-access-8bm4p\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1263e83a-e0fd-4eb4-8141-5026aec6a631-host-slash\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598834    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-cnibin\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598859    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ab05e61-6a22-4ced-abec-a528a15f9ba0-cni-binary-copy\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598887    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-socket-dir-parent\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598914    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1263e83a-e0fd-4eb4-8141-5026aec6a631-host-slash\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598918    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598969    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-hosts-file\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598991    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-cnibin\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.598998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-env-overrides\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.599081    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1263e83a-e0fd-4eb4-8141-5026aec6a631-iptables-alerter-script\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.599384    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ab05e61-6a22-4ced-abec-a528a15f9ba0-cni-binary-copy\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.599489    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-socket-dir-parent\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.599574 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.599537    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-systemd\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.599616    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysconfig\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:16.599646    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:16.599723    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs podName:2758dbad-1f8a-4503-9dc2-6181518daeaf nodeName:}" failed. No retries permitted until 2026-10-30 12:35:17.099701123 +0000 UTC m=+3.139404820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs") pod "network-metrics-daemon-xdcfv" (UID: "2758dbad-1f8a-4503-9dc2-6181518daeaf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.599652    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a594de5b-9748-4355-86b8-95ca7dc9dda1-serviceca\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.599831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lshc4\" (UniqueName: \"kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4\") pod \"network-check-target-29mpd\" (UID: \"a7e45249-4b17-415f-966d-af5321f182d5\") " pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.599863    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-kubelet\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.599889    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-run-netns\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.599912    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovnkube-config\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.599938    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-system-cni-dir\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.599963    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-hostroot\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.599986    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-conf-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600011    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6wnt\" (UniqueName: \"kubernetes.io/projected/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-kube-api-access-q6wnt\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600061    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysconfig\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-ovn\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600126    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a594de5b-9748-4355-86b8-95ca7dc9dda1-serviceca\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600152    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-node-log\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.600408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600131    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-node-log\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600181    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-ovn\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600192    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovnkube-script-lib\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600226    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-run-netns\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600227    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-modprobe-d\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600256    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysctl-d\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600285    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysctl-conf\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600336    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-system-cni-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600365    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600385    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-hosts-file\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600395    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhm96\" (UniqueName: \"kubernetes.io/projected/4b56b071-d626-4d8d-bba6-f5f69fa95db0-kube-api-access-vhm96\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600413    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-kubelet\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600424    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d31facfa-368e-4417-b9ce-30c3a495f650-etc-tuned\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600449    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-hostroot\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600504    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-system-cni-dir\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600511    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-system-cni-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600451    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-825sw\" (UniqueName: \"kubernetes.io/projected/1263e83a-e0fd-4eb4-8141-5026aec6a631-kube-api-access-825sw\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600554    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a594de5b-9748-4355-86b8-95ca7dc9dda1-host\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:16.601249 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600582    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkj5v\" (UniqueName: \"kubernetes.io/projected/a594de5b-9748-4355-86b8-95ca7dc9dda1-kube-api-access-bkj5v\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600607    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-cni-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600624    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-modprobe-d\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600509    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysctl-conf\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600705    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a594de5b-9748-4355-86b8-95ca7dc9dda1-host\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600814    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-cni-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600842    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysctl-d\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600865    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-netns\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.600333    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-env-overrides\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601025    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601062    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovnkube-script-lib\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601086    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-netns\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601132    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-slash\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601140    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-conf-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601161    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601189    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-slash\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601188    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-cni-netd\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601225    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-cni-netd\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.602071 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601230    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-host\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-multus-certs\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601289    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-etc-kubernetes\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601315    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8xqw\" (UniqueName: \"kubernetes.io/projected/2758dbad-1f8a-4503-9dc2-6181518daeaf-kube-api-access-q8xqw\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601337    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-multus-certs\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601377    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-etc-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601377    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-etc-kubernetes\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-etc-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601426    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cni-binary-copy\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601450    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601476    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601499    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-sys\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601524    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-daemon-config\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-var-lib-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601595    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-var-lib-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601606    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-host\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.603022 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601640    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-run\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601667    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-var-lib-kubelet\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601714    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovnkube-config\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601723    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-socket-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601784    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601811    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cnibin\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601824    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-socket-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601837    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-cni-bin\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601864    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-lib-modules\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601873    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-run\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmn28\" (UniqueName: \"kubernetes.io/projected/e937e696-f801-42e7-9622-d774cca50028-kube-api-access-pmn28\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601908    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-var-lib-kubelet\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601920    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-tmp-dir\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601932    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cnibin\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601945    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-kubernetes\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601972    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce-konnectivity-ca\") pod \"konnectivity-agent-p2z49\" (UID: \"7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce\") " pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:16.603904 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601986    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-sys\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601998    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-systemd-units\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.602024    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-cni-bin\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.602049    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/15bb3274-f845-4851-90ae-f4ab08e685c8-kubelet-config\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.602073    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/15bb3274-f845-4851-90ae-f4ab08e685c8-dbus\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.602100    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f86qq\" (UniqueName: \"kubernetes.io/projected/3ab05e61-6a22-4ced-abec-a528a15f9ba0-kube-api-access-f86qq\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.602122    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-sys-fs\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.602168    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cni-binary-copy\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.602207    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-sys-fs\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:16.602290    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:16.602341    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret podName:15bb3274-f845-4851-90ae-f4ab08e685c8 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:17.102321912 +0000 UTC m=+3.142025593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret") pod "global-pull-secret-syncer-ntzbv" (UID: "15bb3274-f845-4851-90ae-f4ab08e685c8") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.602390    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-systemd-units\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.602438    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-cni-bin\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.602454    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-cni-bin\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.602783    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-kubernetes\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.601947    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.602888    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/15bb3274-f845-4851-90ae-f4ab08e685c8-dbus\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:16.604736 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.602953    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/15bb3274-f845-4851-90ae-f4ab08e685c8-kubelet-config\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:16.605577 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.603556    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.605577 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.603729    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.605577 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.603769    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-daemon-config\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.605577 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.604109    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-tmp-dir\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:16.605577 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.604301    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce-konnectivity-ca\") pod \"konnectivity-agent-p2z49\" (UID: \"7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce\") " pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:16.605577 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.605343    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovn-node-metrics-cert\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.605956 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.605938    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d31facfa-368e-4417-b9ce-30c3a495f650-etc-tuned\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.606222 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.606202    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce-agent-certs\") pod \"konnectivity-agent-p2z49\" (UID: \"7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce\") " pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:16.608060 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.608003    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d31facfa-368e-4417-b9ce-30c3a495f650-tmp\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.621767 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.621737    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6wnt\" (UniqueName: \"kubernetes.io/projected/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-kube-api-access-q6wnt\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:16.622270 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.622216    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppcp2\" (UniqueName: \"kubernetes.io/projected/18fbde11-0050-4b55-bc23-a4e5e76ddc81-kube-api-access-ppcp2\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.622375 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.622359    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bm4p\" (UniqueName: \"kubernetes.io/projected/d31facfa-368e-4417-b9ce-30c3a495f650-kube-api-access-8bm4p\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.622822 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:16.622427    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:35:16.622822 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:16.622473    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:35:16.622822 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:16.622489    2581 projected.go:196] Error preparing data for projected volume kube-api-access-lshc4 for pod openshift-network-diagnostics/network-check-target-29mpd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:35:16.622822 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:16.622564    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4 podName:a7e45249-4b17-415f-966d-af5321f182d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:17.122537606 +0000 UTC m=+3.162241298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lshc4" (UniqueName: "kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4") pod "network-check-target-29mpd" (UID: "a7e45249-4b17-415f-966d-af5321f182d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:35:16.626528 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.626505    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8xqw\" (UniqueName: \"kubernetes.io/projected/2758dbad-1f8a-4503-9dc2-6181518daeaf-kube-api-access-q8xqw\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:16.626976 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.626952    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkj5v\" (UniqueName: \"kubernetes.io/projected/a594de5b-9748-4355-86b8-95ca7dc9dda1-kube-api-access-bkj5v\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:16.627359 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.627336    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhm96\" (UniqueName: \"kubernetes.io/projected/4b56b071-d626-4d8d-bba6-f5f69fa95db0-kube-api-access-vhm96\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.630363 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.630335    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmn28\" (UniqueName: \"kubernetes.io/projected/e937e696-f801-42e7-9622-d774cca50028-kube-api-access-pmn28\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.630363 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.630356    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f86qq\" (UniqueName: \"kubernetes.io/projected/3ab05e61-6a22-4ced-abec-a528a15f9ba0-kube-api-access-f86qq\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.631229 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.631205    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-825sw\" (UniqueName: \"kubernetes.io/projected/1263e83a-e0fd-4eb4-8141-5026aec6a631-kube-api-access-825sw\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:16.736540 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.736505    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:16.739677 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.739649    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:16.745832 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.745797    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:16.746035 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.746008    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:16.751203 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:16.751173    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d7b7ac0_cec8_4b06_8bb9_ef5a458f42ce.slice/crio-9ed57772b7f777bf45d80ea1035ae4a8c4e3fc0876275655ceda617b1849ba7f WatchSource:0}: Error finding container 9ed57772b7f777bf45d80ea1035ae4a8c4e3fc0876275655ceda617b1849ba7f: Status 404 returned error can't find the container with id 9ed57772b7f777bf45d80ea1035ae4a8c4e3fc0876275655ceda617b1849ba7f
Oct 30 12:35:16.752425 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:16.752401    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b56b071_d626_4d8d_bba6_f5f69fa95db0.slice/crio-621ace500d4f75ef9c887619ca34581f60018c1b26a94ae2a4dcaf17ceb7fa74 WatchSource:0}: Error finding container 621ace500d4f75ef9c887619ca34581f60018c1b26a94ae2a4dcaf17ceb7fa74: Status 404 returned error can't find the container with id 621ace500d4f75ef9c887619ca34581f60018c1b26a94ae2a4dcaf17ceb7fa74
Oct 30 12:35:16.752519 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.752483    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:16.758630 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:16.758587    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd31facfa_368e_4417_b9ce_30c3a495f650.slice/crio-4e0e5ec5af826df541dfab86ded99fbc774263e9411f9361db30b3e2fa8c12e7 WatchSource:0}: Error finding container 4e0e5ec5af826df541dfab86ded99fbc774263e9411f9361db30b3e2fa8c12e7: Status 404 returned error can't find the container with id 4e0e5ec5af826df541dfab86ded99fbc774263e9411f9361db30b3e2fa8c12e7
Oct 30 12:35:16.760731 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.760712    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:16.767229 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:16.766727    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda594de5b_9748_4355_86b8_95ca7dc9dda1.slice/crio-62c6703912aecec773d95240640baeb4395beeab94b0ac8e1d379492343ac48c WatchSource:0}: Error finding container 62c6703912aecec773d95240640baeb4395beeab94b0ac8e1d379492343ac48c: Status 404 returned error can't find the container with id 62c6703912aecec773d95240640baeb4395beeab94b0ac8e1d379492343ac48c
Oct 30 12:35:16.775364 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.775335    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:16.775736 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:16.775687    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ab05e61_6a22_4ced_abec_a528a15f9ba0.slice/crio-3cd6393cdf0c12b4dadc6f89839a211107470255881edd2e736a1dfef317e7fd WatchSource:0}: Error finding container 3cd6393cdf0c12b4dadc6f89839a211107470255881edd2e736a1dfef317e7fd: Status 404 returned error can't find the container with id 3cd6393cdf0c12b4dadc6f89839a211107470255881edd2e736a1dfef317e7fd
Oct 30 12:35:16.777945 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.777925    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:16.787320 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:16.787284    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode937e696_f801_42e7_9622_d774cca50028.slice/crio-757c56165d7a422d29db3395a4755e5bc5c7251c3dadac21fa6fb98b3fe8cca9 WatchSource:0}: Error finding container 757c56165d7a422d29db3395a4755e5bc5c7251c3dadac21fa6fb98b3fe8cca9: Status 404 returned error can't find the container with id 757c56165d7a422d29db3395a4755e5bc5c7251c3dadac21fa6fb98b3fe8cca9
Oct 30 12:35:16.801452 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:16.801426    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:16.812269 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:16.812240    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18fbde11_0050_4b55_bc23_a4e5e76ddc81.slice/crio-8a9cb48c43d1cf1ab491ef872fe88365a0da7a311fa8f4bb318400164ef97043 WatchSource:0}: Error finding container 8a9cb48c43d1cf1ab491ef872fe88365a0da7a311fa8f4bb318400164ef97043: Status 404 returned error can't find the container with id 8a9cb48c43d1cf1ab491ef872fe88365a0da7a311fa8f4bb318400164ef97043
Oct 30 12:35:17.106232 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.106198    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:17.106364 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.106299    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:17.106430 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:17.106374    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:35:17.106483 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:17.106433    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:35:17.106483 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:17.106451    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs podName:2758dbad-1f8a-4503-9dc2-6181518daeaf nodeName:}" failed. No retries permitted until 2026-10-30 12:35:18.106428131 +0000 UTC m=+4.146131802 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs") pod "network-metrics-daemon-xdcfv" (UID: "2758dbad-1f8a-4503-9dc2-6181518daeaf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:35:17.106595 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:17.106494    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret podName:15bb3274-f845-4851-90ae-f4ab08e685c8 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:18.106474953 +0000 UTC m=+4.146178640 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret") pod "global-pull-secret-syncer-ntzbv" (UID: "15bb3274-f845-4851-90ae-f4ab08e685c8") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:35:17.207451 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.207395    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lshc4\" (UniqueName: \"kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4\") pod \"network-check-target-29mpd\" (UID: \"a7e45249-4b17-415f-966d-af5321f182d5\") " pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:17.207649 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:17.207582    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:35:17.207649 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:17.207603    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:35:17.207649 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:17.207616    2581 projected.go:196] Error preparing data for projected volume kube-api-access-lshc4 for pod openshift-network-diagnostics/network-check-target-29mpd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:35:17.207808 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:17.207706    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4 podName:a7e45249-4b17-415f-966d-af5321f182d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:18.207686018 +0000 UTC m=+4.247389693 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lshc4" (UniqueName: "kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4") pod "network-check-target-29mpd" (UID: "a7e45249-4b17-415f-966d-af5321f182d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:35:17.450337 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.450092    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 12:30:17 +0000 UTC" deadline="2028-07-28 00:04:34.053898076 +0000 UTC"
Oct 30 12:35:17.450999 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.450339    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15275h29m16.603564559s"
Oct 30 12:35:17.514739 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.514456    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:17.514739 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.514698    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:17.514967 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:17.514784    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:17.514967 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.514853    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:17.514967 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:17.514922    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:17.515141 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:17.514610    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:17.538713 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.532237    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kqrtg" event={"ID":"3ab05e61-6a22-4ced-abec-a528a15f9ba0","Type":"ContainerStarted","Data":"3cd6393cdf0c12b4dadc6f89839a211107470255881edd2e736a1dfef317e7fd"}
Oct 30 12:35:17.538713 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.534532    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cpq89" event={"ID":"4b56b071-d626-4d8d-bba6-f5f69fa95db0","Type":"ContainerStarted","Data":"621ace500d4f75ef9c887619ca34581f60018c1b26a94ae2a4dcaf17ceb7fa74"}
Oct 30 12:35:17.555860 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.545735    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal" event={"ID":"45b9204b780ab5797493be0f231a6750","Type":"ContainerStarted","Data":"7d5cdf1ce80b70633707b99762a40ccac4b90fdc7a66be0683724f90c174a6c5"}
Oct 30 12:35:17.570504 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.570420    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal" event={"ID":"6766e4bd763f65c3c761bb65d5e5c301","Type":"ContainerStarted","Data":"6a351f52b540041bd424a6987c8fd0328f5be95f20254374632ac74f48d39427"}
Oct 30 12:35:17.575501 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.575427    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh" event={"ID":"e937e696-f801-42e7-9622-d774cca50028","Type":"ContainerStarted","Data":"757c56165d7a422d29db3395a4755e5bc5c7251c3dadac21fa6fb98b3fe8cca9"}
Oct 30 12:35:17.582290 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.582265    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pt7kp" event={"ID":"a594de5b-9748-4355-86b8-95ca7dc9dda1","Type":"ContainerStarted","Data":"62c6703912aecec773d95240640baeb4395beeab94b0ac8e1d379492343ac48c"}
Oct 30 12:35:17.591311 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.591280    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k9987" event={"ID":"1263e83a-e0fd-4eb4-8141-5026aec6a631","Type":"ContainerStarted","Data":"09fceefd9630cf2a164a77f3bb0545dadde776dfacf0dc5e986919d2ee3116c5"}
Oct 30 12:35:17.594039 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.593984    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5frn2" event={"ID":"d31facfa-368e-4417-b9ce-30c3a495f650","Type":"ContainerStarted","Data":"4e0e5ec5af826df541dfab86ded99fbc774263e9411f9361db30b3e2fa8c12e7"}
Oct 30 12:35:17.596898 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.596839    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p2z49" event={"ID":"7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce","Type":"ContainerStarted","Data":"9ed57772b7f777bf45d80ea1035ae4a8c4e3fc0876275655ceda617b1849ba7f"}
Oct 30 12:35:17.603301 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.603271    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv" event={"ID":"18fbde11-0050-4b55-bc23-a4e5e76ddc81","Type":"ContainerStarted","Data":"8a9cb48c43d1cf1ab491ef872fe88365a0da7a311fa8f4bb318400164ef97043"}
Oct 30 12:35:17.609570 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:17.609540    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cwcj7" event={"ID":"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0","Type":"ContainerStarted","Data":"3351f698fa606515af8dcb112bf4cea5cd1434a7232fdb021c7abf39525e78b1"}
Oct 30 12:35:18.115633 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:18.115589    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:18.115846 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:18.115687    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:18.115846 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:18.115745    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:35:18.115846 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:18.115830    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:35:18.115846 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:18.115839    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret podName:15bb3274-f845-4851-90ae-f4ab08e685c8 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:20.115817195 +0000 UTC m=+6.155520885 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret") pod "global-pull-secret-syncer-ntzbv" (UID: "15bb3274-f845-4851-90ae-f4ab08e685c8") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:35:18.116094 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:18.115889    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs podName:2758dbad-1f8a-4503-9dc2-6181518daeaf nodeName:}" failed. No retries permitted until 2026-10-30 12:35:20.115871557 +0000 UTC m=+6.155575248 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs") pod "network-metrics-daemon-xdcfv" (UID: "2758dbad-1f8a-4503-9dc2-6181518daeaf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:35:18.216923 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:18.216882    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lshc4\" (UniqueName: \"kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4\") pod \"network-check-target-29mpd\" (UID: \"a7e45249-4b17-415f-966d-af5321f182d5\") " pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:18.217131 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:18.217065    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:35:18.217131 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:18.217086    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:35:18.217131 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:18.217107    2581 projected.go:196] Error preparing data for projected volume kube-api-access-lshc4 for pod openshift-network-diagnostics/network-check-target-29mpd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:35:18.217305 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:18.217169    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4 podName:a7e45249-4b17-415f-966d-af5321f182d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:20.217150707 +0000 UTC m=+6.256854391 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lshc4" (UniqueName: "kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4") pod "network-check-target-29mpd" (UID: "a7e45249-4b17-415f-966d-af5321f182d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:35:18.450967 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:18.450824    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 12:30:17 +0000 UTC" deadline="2028-08-14 03:14:23.074129618 +0000 UTC"
Oct 30 12:35:18.450967 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:18.450885    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15686h39m4.623248924s"
Oct 30 12:35:18.629393 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:18.629354    2581 generic.go:356] "Generic (PLEG): container finished" podID="45b9204b780ab5797493be0f231a6750" containerID="7d5cdf1ce80b70633707b99762a40ccac4b90fdc7a66be0683724f90c174a6c5" exitCode=0
Oct 30 12:35:18.630522 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:18.630494    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal" event={"ID":"45b9204b780ab5797493be0f231a6750","Type":"ContainerDied","Data":"7d5cdf1ce80b70633707b99762a40ccac4b90fdc7a66be0683724f90c174a6c5"}
Oct 30 12:35:18.671971 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:18.671906    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal" podStartSLOduration=2.6718852440000003 podStartE2EDuration="2.671885244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:35:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:17.621416087 +0000 UTC m=+3.661119789" watchObservedRunningTime="2026-10-30 12:35:18.671885244 +0000 UTC m=+4.711588942"
Oct 30 12:35:19.514791 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:19.514687    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:19.514791 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:19.514725    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:19.515256 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:19.514832    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:19.515316 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:19.515260    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:19.515368 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:19.515323    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:19.515424 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:19.515404    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:19.637559 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:19.636615    2581 generic.go:356] "Generic (PLEG): container finished" podID="4b56b071-d626-4d8d-bba6-f5f69fa95db0" containerID="31ce6209c88915a87e9720f4ce8a2b0483c3d61710c08081de978b881fc89a5a" exitCode=0
Oct 30 12:35:19.637559 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:19.636723    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cpq89" event={"ID":"4b56b071-d626-4d8d-bba6-f5f69fa95db0","Type":"ContainerDied","Data":"31ce6209c88915a87e9720f4ce8a2b0483c3d61710c08081de978b881fc89a5a"}
Oct 30 12:35:19.642863 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:19.642287    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal" event={"ID":"45b9204b780ab5797493be0f231a6750","Type":"ContainerStarted","Data":"96909b7980c00dee51ef2280b27268d3dea44ed88dd69a42df305f4aaf6f2346"}
Oct 30 12:35:20.133164 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:20.133125    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:20.133340 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:20.133201    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:20.133399 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:20.133366    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:35:20.133473 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:20.133427    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs podName:2758dbad-1f8a-4503-9dc2-6181518daeaf nodeName:}" failed. No retries permitted until 2026-10-30 12:35:24.13340919 +0000 UTC m=+10.173112882 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs") pod "network-metrics-daemon-xdcfv" (UID: "2758dbad-1f8a-4503-9dc2-6181518daeaf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:35:20.133880 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:20.133860    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:35:20.133971 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:20.133921    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret podName:15bb3274-f845-4851-90ae-f4ab08e685c8 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:24.133905284 +0000 UTC m=+10.173608972 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret") pod "global-pull-secret-syncer-ntzbv" (UID: "15bb3274-f845-4851-90ae-f4ab08e685c8") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:35:20.234402 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:20.234364    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lshc4\" (UniqueName: \"kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4\") pod \"network-check-target-29mpd\" (UID: \"a7e45249-4b17-415f-966d-af5321f182d5\") " pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:20.234560 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:20.234543    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:35:20.234635 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:20.234563    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:35:20.234635 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:20.234577    2581 projected.go:196] Error preparing data for projected volume kube-api-access-lshc4 for pod openshift-network-diagnostics/network-check-target-29mpd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:35:20.234775 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:20.234661    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4 podName:a7e45249-4b17-415f-966d-af5321f182d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:24.234621523 +0000 UTC m=+10.274325216 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lshc4" (UniqueName: "kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4") pod "network-check-target-29mpd" (UID: "a7e45249-4b17-415f-966d-af5321f182d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:35:21.514379 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:21.514341    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:21.514955 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:21.514496    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:21.514955 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:21.514935    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:21.515129 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:21.515091    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:21.515202 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:21.515159    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:21.515265 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:21.515227    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:21.651429 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:21.651393    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p2z49" event={"ID":"7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce","Type":"ContainerStarted","Data":"deadc1e9ccb708520f035d26b4bfeefc3a7b74a30e7c421361e17e650b86fb9e"}
Oct 30 12:35:21.691245 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:21.691190    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal" podStartSLOduration=5.691175781 podStartE2EDuration="5.691175781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:35:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:19.749886165 +0000 UTC m=+5.789589862" watchObservedRunningTime="2026-10-30 12:35:21.691175781 +0000 UTC m=+7.730879475"
Oct 30 12:35:21.691729 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:21.691688    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-p2z49" podStartSLOduration=2.840278553 podStartE2EDuration="6.69165399s" totalImagesPullingTime="3.851375437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:21.691391471 +0000 UTC m=+7.731095165" watchObservedRunningTime="2026-10-30 12:35:21.69165399 +0000 UTC m=+7.731357685"
Oct 30 12:35:22.657113 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:22.657076    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pt7kp" event={"ID":"a594de5b-9748-4355-86b8-95ca7dc9dda1","Type":"ContainerStarted","Data":"9ddf9c9bb8d9ed042097d53f2cdb8a82f18fcab27b5ef326122b8f790747efa8"}
Oct 30 12:35:23.514540 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:23.513629    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:23.514540 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:23.513795    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:23.514540 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:23.514224    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:23.514540 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:23.514347    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:23.514540 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:23.514407    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:23.514540 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:23.514480    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:23.664082 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:23.663743    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh" event={"ID":"e937e696-f801-42e7-9622-d774cca50028","Type":"ContainerStarted","Data":"e3d3cb143ad3a8314805999928c55d5acd5c79ce334b6edea1d32eb0af9e157e"}
Oct 30 12:35:24.166868 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:24.165748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:24.166868 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:24.165838    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:24.166868 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:24.165960    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:35:24.166868 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:24.166030    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret podName:15bb3274-f845-4851-90ae-f4ab08e685c8 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:32.166010127 +0000 UTC m=+18.205713817 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret") pod "global-pull-secret-syncer-ntzbv" (UID: "15bb3274-f845-4851-90ae-f4ab08e685c8") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:35:24.166868 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:24.166107    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:35:24.166868 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:24.166144    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs podName:2758dbad-1f8a-4503-9dc2-6181518daeaf nodeName:}" failed. No retries permitted until 2026-10-30 12:35:32.166131815 +0000 UTC m=+18.205835497 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs") pod "network-metrics-daemon-xdcfv" (UID: "2758dbad-1f8a-4503-9dc2-6181518daeaf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:35:24.266661 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:24.266343    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lshc4\" (UniqueName: \"kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4\") pod \"network-check-target-29mpd\" (UID: \"a7e45249-4b17-415f-966d-af5321f182d5\") " pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:24.266661 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:24.266522    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:35:24.266661 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:24.266548    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:35:24.266661 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:24.266569    2581 projected.go:196] Error preparing data for projected volume kube-api-access-lshc4 for pod openshift-network-diagnostics/network-check-target-29mpd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:35:24.266661 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:24.266652    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4 podName:a7e45249-4b17-415f-966d-af5321f182d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:32.266630708 +0000 UTC m=+18.306334378 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lshc4" (UniqueName: "kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4") pod "network-check-target-29mpd" (UID: "a7e45249-4b17-415f-966d-af5321f182d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:35:24.472166 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:24.472091    2581 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 12:35:24.670134 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:24.670043    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh" event={"ID":"e937e696-f801-42e7-9622-d774cca50028","Type":"ContainerStarted","Data":"44a3c5f9ae722f40ea7e1870814b6e03d379be5573ee9693044495dff9d87fde"}
Oct 30 12:35:25.443340 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.442983    2581 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T12:35:24.472106228Z","UUID":"ae611621-75af-4fb9-b761-1ca1bbdaf16d","Handler":null,"Name":"","Endpoint":""}
Oct 30 12:35:25.444530 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.444507    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 12:35:25.444530 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.444535    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 12:35:25.513733 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.513706    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:25.513733 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.513728    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:25.513944 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.513717    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:25.513944 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:25.513821    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:25.513944 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:25.513929    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:25.514138 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:25.514022    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:25.675735 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.675703    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh" event={"ID":"e937e696-f801-42e7-9622-d774cca50028","Type":"ContainerStarted","Data":"01bcaa0e9b1ff208d3868575122ae3137573790a16dadd5d3e082a3e5fbbeb24"}
Oct 30 12:35:25.678838 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.678803    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cwcj7" event={"ID":"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0","Type":"ContainerStarted","Data":"9720d49c1fb37f1f10417daed40085b327afe80274de15280f341361128683a9"}
Oct 30 12:35:25.680226 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.680201    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kqrtg" event={"ID":"3ab05e61-6a22-4ced-abec-a528a15f9ba0","Type":"ContainerStarted","Data":"90edc5c0ced120a50f81d7558a2e79492d6c2e2aaa71c424352d9410e422e07b"}
Oct 30 12:35:25.682271 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.682244    2581 generic.go:356] "Generic (PLEG): container finished" podID="4b56b071-d626-4d8d-bba6-f5f69fa95db0" containerID="9727da6e4443a110df6503922683247d0818f78fdff3cb502b0d171519330a5e" exitCode=0
Oct 30 12:35:25.682453 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.682281    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cpq89" event={"ID":"4b56b071-d626-4d8d-bba6-f5f69fa95db0","Type":"ContainerDied","Data":"9727da6e4443a110df6503922683247d0818f78fdff3cb502b0d171519330a5e"}
Oct 30 12:35:25.716425 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.716319    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh" podStartSLOduration=2.3852089149999998 podStartE2EDuration="10.716304558s" totalImagesPullingTime="8.331095643s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:35:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:25.715849432 +0000 UTC m=+11.755553137" watchObservedRunningTime="2026-10-30 12:35:25.716304558 +0000 UTC m=+11.756008248"
Oct 30 12:35:25.716681 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.716638    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pt7kp" podStartSLOduration=5.399421558 podStartE2EDuration="10.71663004s" totalImagesPullingTime="5.317208482s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:22.69718921 +0000 UTC m=+8.736892909" watchObservedRunningTime="2026-10-30 12:35:25.71663004 +0000 UTC m=+11.756333733"
Oct 30 12:35:25.847257 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.847223    2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:25.847473 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.847421    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-cwcj7" podStartSLOduration=2.504989057 podStartE2EDuration="10.847404005s" totalImagesPullingTime="8.342414948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:25.84709821 +0000 UTC m=+11.886801901" watchObservedRunningTime="2026-10-30 12:35:25.847404005 +0000 UTC m=+11.887107906"
Oct 30 12:35:25.847932 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.847791    2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:25.847932 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:25.847844    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-kqrtg" podStartSLOduration=2.498981687 podStartE2EDuration="10.847831654s" totalImagesPullingTime="8.348849967s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:25.807003835 +0000 UTC m=+11.846707527" watchObservedRunningTime="2026-10-30 12:35:25.847831654 +0000 UTC m=+11.887535346"
Oct 30 12:35:26.686703 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:26.686623    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k9987" event={"ID":"1263e83a-e0fd-4eb4-8141-5026aec6a631","Type":"ContainerStarted","Data":"3777252b06a407bc3643440cd2a697747f9e2d9672757ae882a7be10352c9179"}
Oct 30 12:35:26.689100 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:26.689058    2581 generic.go:356] "Generic (PLEG): container finished" podID="4b56b071-d626-4d8d-bba6-f5f69fa95db0" containerID="57afecd87278bf3ee190f26209e7b0009092ffcf2e4ddff462c911e4a2a9d2b6" exitCode=0
Oct 30 12:35:26.689227 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:26.689126    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cpq89" event={"ID":"4b56b071-d626-4d8d-bba6-f5f69fa95db0","Type":"ContainerDied","Data":"57afecd87278bf3ee190f26209e7b0009092ffcf2e4ddff462c911e4a2a9d2b6"}
Oct 30 12:35:26.722971 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:26.722919    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-k9987" podStartSLOduration=3.523096183 podStartE2EDuration="11.722901163s" totalImagesPullingTime="8.19980498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:26.722692422 +0000 UTC m=+12.762396140" watchObservedRunningTime="2026-10-30 12:35:26.722901163 +0000 UTC m=+12.762604858"
Oct 30 12:35:27.514316 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:27.514290    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:27.514417 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:27.514320    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:27.514486 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:27.514415    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:27.514542 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:27.514493    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:27.514542 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:27.514535    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:27.514693 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:27.514676    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:27.692507 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:27.692469    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5frn2" event={"ID":"d31facfa-368e-4417-b9ce-30c3a495f650","Type":"ContainerStarted","Data":"548f93bb6fac55a66a235ef30794816f92597e058a83e52bebc83bc80a7dd878"}
Oct 30 12:35:27.695413 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:27.695384    2581 generic.go:356] "Generic (PLEG): container finished" podID="4b56b071-d626-4d8d-bba6-f5f69fa95db0" containerID="fa73653c08ffe682b691c171199ba34b881bcf0e65e66ef666ebfd4867a2f74b" exitCode=0
Oct 30 12:35:27.695524 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:27.695464    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cpq89" event={"ID":"4b56b071-d626-4d8d-bba6-f5f69fa95db0","Type":"ContainerDied","Data":"fa73653c08ffe682b691c171199ba34b881bcf0e65e66ef666ebfd4867a2f74b"}
Oct 30 12:35:27.730373 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:27.730332    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5frn2" podStartSLOduration=2.219240718 podStartE2EDuration="12.73031983s" totalImagesPullingTime="10.511079112s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:27.72983754 +0000 UTC m=+13.769541231" watchObservedRunningTime="2026-10-30 12:35:27.73031983 +0000 UTC m=+13.770023520"
Oct 30 12:35:29.514397 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:29.514366    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:29.514397 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:29.514403    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:29.515002 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:29.514366    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:29.515002 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:29.514498    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:29.515002 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:29.514559    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:29.515002 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:29.514648    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:29.703250 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:29.702790    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cpq89" event={"ID":"4b56b071-d626-4d8d-bba6-f5f69fa95db0","Type":"ContainerStarted","Data":"b86e9c179bdd5a1c6ff66dc564b9a6a9ad6b3ca46e3f9731f010bb2ef1e5964a"}
Oct 30 12:35:30.708255 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:30.708224    2581 generic.go:356] "Generic (PLEG): container finished" podID="4b56b071-d626-4d8d-bba6-f5f69fa95db0" containerID="b86e9c179bdd5a1c6ff66dc564b9a6a9ad6b3ca46e3f9731f010bb2ef1e5964a" exitCode=0
Oct 30 12:35:30.708670 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:30.708261    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cpq89" event={"ID":"4b56b071-d626-4d8d-bba6-f5f69fa95db0","Type":"ContainerDied","Data":"b86e9c179bdd5a1c6ff66dc564b9a6a9ad6b3ca46e3f9731f010bb2ef1e5964a"}
Oct 30 12:35:31.514086 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:31.514055    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:31.514086 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:31.514087    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:31.514296 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:31.514150    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:31.514296 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:31.514183    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:31.514296 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:31.514260    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:31.514421 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:31.514349    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:31.712527 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:31.712500    2581 generic.go:356] "Generic (PLEG): container finished" podID="4b56b071-d626-4d8d-bba6-f5f69fa95db0" containerID="c7b150ab472af683719dd2a5651cbb505b582e7e155eb5473f1fe723126e4d73" exitCode=0
Oct 30 12:35:31.712982 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:31.712551    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cpq89" event={"ID":"4b56b071-d626-4d8d-bba6-f5f69fa95db0","Type":"ContainerDied","Data":"c7b150ab472af683719dd2a5651cbb505b582e7e155eb5473f1fe723126e4d73"}
Oct 30 12:35:32.223652 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:32.223471    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:32.223652 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:32.223640    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:35:32.223835 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:32.223684    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:32.223835 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:32.223701    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs podName:2758dbad-1f8a-4503-9dc2-6181518daeaf nodeName:}" failed. No retries permitted until 2026-10-30 12:35:48.223686811 +0000 UTC m=+34.263390479 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs") pod "network-metrics-daemon-xdcfv" (UID: "2758dbad-1f8a-4503-9dc2-6181518daeaf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 12:35:32.223835 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:32.223747    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 12:35:32.223835 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:32.223783    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret podName:15bb3274-f845-4851-90ae-f4ab08e685c8 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:48.223772686 +0000 UTC m=+34.263476370 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret") pod "global-pull-secret-syncer-ntzbv" (UID: "15bb3274-f845-4851-90ae-f4ab08e685c8") : object "kube-system"/"original-pull-secret" not registered
Oct 30 12:35:32.324694 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:32.324620    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lshc4\" (UniqueName: \"kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4\") pod \"network-check-target-29mpd\" (UID: \"a7e45249-4b17-415f-966d-af5321f182d5\") " pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:32.324810 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:32.324745    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 12:35:32.324810 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:32.324759    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 12:35:32.324810 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:32.324767    2581 projected.go:196] Error preparing data for projected volume kube-api-access-lshc4 for pod openshift-network-diagnostics/network-check-target-29mpd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:35:32.324921 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:32.324811    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4 podName:a7e45249-4b17-415f-966d-af5321f182d5 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:48.324797283 +0000 UTC m=+34.364500965 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lshc4" (UniqueName: "kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4") pod "network-check-target-29mpd" (UID: "a7e45249-4b17-415f-966d-af5321f182d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 12:35:32.716965 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:32.716889    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv" event={"ID":"18fbde11-0050-4b55-bc23-a4e5e76ddc81","Type":"ContainerStarted","Data":"a7c9d601a3ed1facd6d3ad9b8a36ea4c689ce02e2037df8e51320e9d3779e69b"}
Oct 30 12:35:32.717350 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:32.716967    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv" event={"ID":"18fbde11-0050-4b55-bc23-a4e5e76ddc81","Type":"ContainerStarted","Data":"fe4febcbfa4dc713fd69507f35779ab0ce3154f6e5bc7821eb1214b7bb2dbe29"}
Oct 30 12:35:32.717350 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:32.717008    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv" event={"ID":"18fbde11-0050-4b55-bc23-a4e5e76ddc81","Type":"ContainerStarted","Data":"75f26436081c946bb0cb44a69606bb98e500f4f4117420fa813579b984b7bc3f"}
Oct 30 12:35:32.717350 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:32.717038    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv" event={"ID":"18fbde11-0050-4b55-bc23-a4e5e76ddc81","Type":"ContainerStarted","Data":"ff905410ddb0b7e3d0b9aa36f7668d14f51473cefa1cb309b9ac7420dc6fb0d0"}
Oct 30 12:35:32.717350 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:32.717068    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv" event={"ID":"18fbde11-0050-4b55-bc23-a4e5e76ddc81","Type":"ContainerStarted","Data":"4f853cb2e7b260c3c8c2ca53a395a72b00594aa5cdcc3015d83cf3f46fd488c5"}
Oct 30 12:35:32.717350 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:32.717113    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv" event={"ID":"18fbde11-0050-4b55-bc23-a4e5e76ddc81","Type":"ContainerStarted","Data":"4b1ad1e6b50b27cf72bba42d64d0c1450bae26a4cf0a276e59b6477a73cb8051"}
Oct 30 12:35:32.720132 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:32.720104    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cpq89" event={"ID":"4b56b071-d626-4d8d-bba6-f5f69fa95db0","Type":"ContainerStarted","Data":"ab5ff3db1b5f32ab9525f980365913aabf1866a1a4ed56c7f2c26b74e34834f3"}
Oct 30 12:35:32.761488 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:32.761450    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cpq89" podStartSLOduration=7.037624767 podStartE2EDuration="17.761439841s" totalImagesPullingTime="10.723815074s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:35:15 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:32.761063168 +0000 UTC m=+18.800766859" watchObservedRunningTime="2026-10-30 12:35:32.761439841 +0000 UTC m=+18.801143532"
Oct 30 12:35:33.513775 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:33.513726    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:33.513775 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:33.513775    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:33.513775 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:33.513785    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:33.514010 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:33.513863    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:33.514010 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:33.513977    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:33.514072 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:33.514059    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:34.067830 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:34.067804    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:34.068278 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:34.067933    2581 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 12:35:34.068375 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:34.068354    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:34.727504 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:34.727468    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv" event={"ID":"18fbde11-0050-4b55-bc23-a4e5e76ddc81","Type":"ContainerStarted","Data":"f4ed63169855b673e41f0b82f9c68e47360d6b1846ebe233d8eb598b5b2e8288"}
Oct 30 12:35:35.514525 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:35.514491    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:35.514525 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:35.514528    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:35.515004 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:35.514585    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:35.515004 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:35.514617    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:35.515004 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:35.514675    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:35.515004 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:35.514729    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:36.735087 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:36.734996    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv" event={"ID":"18fbde11-0050-4b55-bc23-a4e5e76ddc81","Type":"ContainerStarted","Data":"8e3d0fd9c7baf0b214ef20b6a4cbb4c95de361c7647ac5b173a4b4f75db5edb0"}
Oct 30 12:35:36.735931 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:36.735244    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:36.735931 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:36.735311    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:36.735931 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:36.735358    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:36.749795 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:36.749771    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:36.749964 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:36.749951    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:36.814683 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:36.814645    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv" podStartSLOduration=6.805039351 podStartE2EDuration="21.814632524s" totalImagesPullingTime="15.009593173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:36.802429189 +0000 UTC m=+22.842132879" watchObservedRunningTime="2026-10-30 12:35:36.814632524 +0000 UTC m=+22.854336214"
Oct 30 12:35:37.514665 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:37.514628    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:37.514867 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:37.514641    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:37.514867 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:37.514772    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:37.514867 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:37.514763    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:37.514867 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:37.514857    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:37.515064 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:37.514927    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:38.362919 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:38.362891    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ntzbv"]
Oct 30 12:35:38.363378 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:38.363020    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:38.363378 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:38.363123    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:38.365570 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:38.365536    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-29mpd"]
Oct 30 12:35:38.365848 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:38.365828    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:38.365962 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:38.365933    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:38.374456 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:38.374436    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xdcfv"]
Oct 30 12:35:38.374549 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:38.374521    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:38.374636 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:38.374614    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:40.514066 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:40.513901    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:40.514571 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:40.513906    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:40.514571 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:40.514167    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:40.514571 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:40.513906    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:40.514571 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:40.514222    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:40.514571 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:40.514288    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:42.514146 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:42.514116    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:42.514146 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:42.514134    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:42.514562 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:42.514134    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:42.514562 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:42.514214    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:42.514562 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:42.514306    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:42.514562 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:42.514359    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:44.515035 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.515011    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:44.515408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.515172    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:44.515408 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:44.515199    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdcfv" podUID="2758dbad-1f8a-4503-9dc2-6181518daeaf"
Oct 30 12:35:44.515408 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:44.515234    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ntzbv" podUID="15bb3274-f845-4851-90ae-f4ab08e685c8"
Oct 30 12:35:44.515408 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.515274    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:44.515408 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:44.515318    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-29mpd" podUID="a7e45249-4b17-415f-966d-af5321f182d5"
Oct 30 12:35:44.719627 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.719589    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeReady"
Oct 30 12:35:44.719783 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.719777    2581 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 12:35:44.765810 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.765783    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-5lwxr"]
Oct 30 12:35:44.769453 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.769430    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-5lwxr"
Oct 30 12:35:44.770751 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.770727    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-57b7896669-ndwsr"]
Oct 30 12:35:44.773748 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.773726    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-846cf5b65-gllvj"]
Oct 30 12:35:44.773858 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.773844    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:44.778245 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.778223    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"]
Oct 30 12:35:44.778376 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.778354    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:44.780976 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.780953    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-q74lv"]
Oct 30 12:35:44.781097 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.781080    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:44.790640 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.790589    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-q8mz6"]
Oct 30 12:35:44.790987 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.790965    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-q74lv"
Oct 30 12:35:44.792771 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.792753    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 12:35:44.793850 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.793829    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"]
Oct 30 12:35:44.795177 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.794014    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-q8mz6"
Oct 30 12:35:44.798090 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.798055    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 12:35:44.798194 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.798140    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 12:35:44.798194 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.798160    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 12:35:44.798449 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.798433    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 12:35:44.798972 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.798711    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vtqg8\""
Oct 30 12:35:44.802875 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.799630    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-m6xzg"]
Oct 30 12:35:44.802875 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.799793    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:44.802875 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.802431    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"]
Oct 30 12:35:44.802875 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.802540    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:44.806971 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.806953    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"]
Oct 30 12:35:44.807095 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.807074    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:44.813093 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.810088    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6mt4g"]
Oct 30 12:35:44.813093 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.811083    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:44.819861 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.819841    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"]
Oct 30 12:35:44.820713 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.820692    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6mt4g"
Oct 30 12:35:44.824562 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.824012    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f558d755b-bj4vz"]
Oct 30 12:35:44.824562 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.824332    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:44.829996 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.829974    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-44v66"]
Oct 30 12:35:44.831710 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.831689    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:44.836569 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.836551    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-gzkb8"]
Oct 30 12:35:44.836937 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.836708    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:44.840450 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.840431    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2djq9"]
Oct 30 12:35:44.840679 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.840658    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-gzkb8"
Oct 30 12:35:44.846895 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.846831    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-t5j98"]
Oct 30 12:35:44.847002 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.846946    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:44.850865 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.850844    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-5lwxr"]
Oct 30 12:35:44.850968 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.850881    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57b7896669-ndwsr"]
Oct 30 12:35:44.850968 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.850903    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-q74lv"]
Oct 30 12:35:44.850968 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.850925    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"]
Oct 30 12:35:44.850968 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.850945    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-q8mz6"]
Oct 30 12:35:44.850968 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.850966    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"]
Oct 30 12:35:44.851174 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.850987    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"]
Oct 30 12:35:44.851174 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.851006    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-m6xzg"]
Oct 30 12:35:44.851174 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.851024    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"]
Oct 30 12:35:44.851174 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.851044    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"]
Oct 30 12:35:44.851174 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.851077    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-gzkb8"]
Oct 30 12:35:44.851174 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.851104    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-846cf5b65-gllvj"]
Oct 30 12:35:44.851174 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.851124    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-44v66"]
Oct 30 12:35:44.851174 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.851145    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2djq9"]
Oct 30 12:35:44.851174 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.851172    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6mt4g"]
Oct 30 12:35:44.851455 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.851192    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t5j98"]
Oct 30 12:35:44.851455 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.851208    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f558d755b-bj4vz"]
Oct 30 12:35:44.851455 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.851256    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t5j98"
Oct 30 12:35:44.897208 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.897173    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 12:35:44.897827 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.897808    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 12:35:44.897944 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.897828    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 12:35:44.897944 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.897837    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 12:35:44.898109 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.897858    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 12:35:44.898109 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.897888    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 12:35:44.898109 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.897895    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 12:35:44.898109 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.897899    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 12:35:44.898109 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.897904    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 12:35:44.898109 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.897986    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 12:35:44.898109 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.897995    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 12:35:44.898109 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898025    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:44.898109 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898044    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 12:35:44.898109 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898051    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 12:35:44.898109 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898058    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hqrrz\""
Oct 30 12:35:44.898109 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898062    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 12:35:44.898109 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898107    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 12:35:44.898725 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898120    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 12:35:44.898725 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898128    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 12:35:44.898725 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898237    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 12:35:44.898725 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898239    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8phcg\""
Oct 30 12:35:44.898725 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898276    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tz7hb\""
Oct 30 12:35:44.898725 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898317    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-vwpvk\""
Oct 30 12:35:44.898725 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898368    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-nnzn8\""
Oct 30 12:35:44.898725 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898481    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:44.898725 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898481    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qgn2s\""
Oct 30 12:35:44.898725 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898555    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-8r6jz\""
Oct 30 12:35:44.899286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898742    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:44.899286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.898913    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:44.899286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.899086    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 12:35:44.899286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.899223    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 12:35:44.899286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.899247    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 12:35:44.908800 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.908782    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hpqh5\""
Oct 30 12:35:44.908881 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.908803    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 12:35:44.908977 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.908960    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9zjv6\""
Oct 30 12:35:44.909080 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.909005    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 12:35:44.909080 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.909027    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 12:35:44.909634 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.909585    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 12:35:44.909714 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.909618    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 12:35:44.909714 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.909653    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 12:35:44.909714 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.909653    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:44.909714 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.909688    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 12:35:44.909714 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.909700    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 12:35:44.909951 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.909699    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 12:35:44.909951 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.909832    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 12:35:44.909951 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.909937    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 12:35:44.910114 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.909937    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-c8dt7\""
Oct 30 12:35:44.910114 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.909992    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 12:35:44.910114 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.910070    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 12:35:44.910114 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.910097    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-x7vtx\""
Oct 30 12:35:44.910319 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.910140    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 12:35:44.910499 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.910475    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-pdckv\""
Oct 30 12:35:44.921676 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.921660    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 12:35:44.923283 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.923262    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 12:35:44.923862 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.923828    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:44.924005 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.923989    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5nr5\" (UniqueName: \"kubernetes.io/projected/10337f54-95e9-42cd-a73f-16817fc1cb96-kube-api-access-p5nr5\") pod \"ingress-canary-t5j98\" (UID: \"10337f54-95e9-42cd-a73f-16817fc1cb96\") " pod="openshift-ingress-canary/ingress-canary-t5j98"
Oct 30 12:35:44.924139 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.924124    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2khhs\" (UniqueName: \"kubernetes.io/projected/2be5910f-12fa-46fe-bf9a-ddff510f4d09-kube-api-access-2khhs\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:44.924239 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.924223    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2v9l\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-kube-api-access-k2v9l\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:44.924335 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.924322    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzdfs\" (UniqueName: \"kubernetes.io/projected/79f0f110-a04f-4b35-8eba-c8f0469b62a1-kube-api-access-bzdfs\") pod \"kas-connection-checker-685895c7c9-q8mz6\" (UID: \"79f0f110-a04f-4b35-8eba-c8f0469b62a1\") " pod="kube-system/kas-connection-checker-685895c7c9-q8mz6"
Oct 30 12:35:44.924419 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.924406    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-installation-pull-secrets\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:44.924513 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.924500    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-bound-sa-token\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:44.924617 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.924588    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-certificates\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:44.924717 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.924704    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:44.924805 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.924656    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 12:35:44.924880 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.924865    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aba04aa0-2bee-47e2-9f36-8790309d6e06-serving-cert\") pod \"service-ca-operator-544d749c8c-x976j\" (UID: \"aba04aa0-2bee-47e2-9f36-8790309d6e06\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:44.924977 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.924964    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn8lm\" (UniqueName: \"kubernetes.io/projected/3a2f595c-6efe-4b69-81bf-074985a6ddde-kube-api-access-fn8lm\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:44.925063 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.925051    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/60e62efb-7c65-41f3-87d4-0805eec8f7e9-snapshots\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:44.925148 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.925136    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:44.925237 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.925223    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk2ml\" (UniqueName: \"kubernetes.io/projected/60e62efb-7c65-41f3-87d4-0805eec8f7e9-kube-api-access-xk2ml\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:44.925360 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.925347    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-stats-auth\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:44.925460 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.925447    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfwdk\" (UniqueName: \"kubernetes.io/projected/d81730d3-53ed-40b8-a512-348fc042f821-kube-api-access-sfwdk\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:44.925552 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.925540    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/49837f84-8643-4643-aa92-3716a9b755d1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:44.925672 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.925656    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b6xs\" (UniqueName: \"kubernetes.io/projected/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-kube-api-access-9b6xs\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:44.925775 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.925761    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jknfr\" (UniqueName: \"kubernetes.io/projected/10a8a53c-d9d6-45aa-936e-48745236bb01-kube-api-access-jknfr\") pod \"network-check-source-6f78d59f99-gzkb8\" (UID: \"10a8a53c-d9d6-45aa-936e-48745236bb01\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-gzkb8"
Oct 30 12:35:44.925893 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.925781    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 12:35:44.926000 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.925984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-trusted-ca\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:44.926105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.926090    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60e62efb-7c65-41f3-87d4-0805eec8f7e9-tmp\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:44.926221 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.926208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/60e62efb-7c65-41f3-87d4-0805eec8f7e9-serving-cert\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:44.926335 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.926321    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:44.926431 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.926419    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-config\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:44.926542 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.926527    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:44.927183 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.926668    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e089d179-e583-4590-ba4c-39e7d486d707-tmp-dir\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:44.927257 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.927227    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:44.927623 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.927588    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-serving-cert\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:44.927698 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.927641    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e62efb-7c65-41f3-87d4-0805eec8f7e9-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:44.927698 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.927668    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk26b\" (UniqueName: \"kubernetes.io/projected/e089d179-e583-4590-ba4c-39e7d486d707-kube-api-access-rk26b\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:44.927808 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.927754    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:44.927808 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.927782    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4f13aca3-562e-406b-a154-8755b1bf7d15-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w\" (UID: \"4f13aca3-562e-406b-a154-8755b1bf7d15\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:44.927879 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.927857    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3a2f595c-6efe-4b69-81bf-074985a6ddde-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:44.927915 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.927879    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10337f54-95e9-42cd-a73f-16817fc1cb96-cert\") pod \"ingress-canary-t5j98\" (UID: \"10337f54-95e9-42cd-a73f-16817fc1cb96\") " pod="openshift-ingress-canary/ingress-canary-t5j98"
Oct 30 12:35:44.927915 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.927898    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfh24\" (UniqueName: \"kubernetes.io/projected/4f13aca3-562e-406b-a154-8755b1bf7d15-kube-api-access-cfh24\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w\" (UID: \"4f13aca3-562e-406b-a154-8755b1bf7d15\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:44.928011 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.927915    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-trusted-ca\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:44.928011 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.927958    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d7qp\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-kube-api-access-2d7qp\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:44.928011 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.927986    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xftjm\" (UniqueName: \"kubernetes.io/projected/aba04aa0-2bee-47e2-9f36-8790309d6e06-kube-api-access-xftjm\") pod \"service-ca-operator-544d749c8c-x976j\" (UID: \"aba04aa0-2bee-47e2-9f36-8790309d6e06\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:44.928114 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928037    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:44.928114 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928070    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-ca-trust-extracted\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:44.928186 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928145    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-installation-pull-secrets\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:44.928233 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928183    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f13aca3-562e-406b-a154-8755b1bf7d15-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w\" (UID: \"4f13aca3-562e-406b-a154-8755b1bf7d15\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:44.928233 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928208    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e089d179-e583-4590-ba4c-39e7d486d707-config-volume\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:44.928335 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928259    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aba04aa0-2bee-47e2-9f36-8790309d6e06-config\") pod \"service-ca-operator-544d749c8c-x976j\" (UID: \"aba04aa0-2bee-47e2-9f36-8790309d6e06\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:44.928335 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928288    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e62efb-7c65-41f3-87d4-0805eec8f7e9-service-ca-bundle\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:44.928335 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928317    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slh7l\" (UniqueName: \"kubernetes.io/projected/06910518-58db-450f-a5ee-521ac8f51e1a-kube-api-access-slh7l\") pod \"kas-connection-checker-685895c7c9-q74lv\" (UID: \"06910518-58db-450f-a5ee-521ac8f51e1a\") " pod="kube-system/kas-connection-checker-685895c7c9-q74lv"
Oct 30 12:35:44.928437 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928371    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-default-certificate\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:44.928437 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928396    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:44.928437 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928430    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-certificates\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:44.928559 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928452    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-bound-sa-token\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:44.928559 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928505    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fbjv\" (UniqueName: \"kubernetes.io/projected/336de054-d674-4f26-81d8-980b11e27ace-kube-api-access-5fbjv\") pod \"volume-data-source-validator-58c9f5f46d-6mt4g\" (UID: \"336de054-d674-4f26-81d8-980b11e27ace\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6mt4g"
Oct 30 12:35:44.928559 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928544    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52p8s\" (UniqueName: \"kubernetes.io/projected/e30d5797-e034-4806-be4f-54f54698920c-kube-api-access-52p8s\") pod \"kas-connection-checker-685895c7c9-5lwxr\" (UID: \"e30d5797-e034-4806-be4f-54f54698920c\") " pod="kube-system/kas-connection-checker-685895c7c9-5lwxr"
Oct 30 12:35:44.928702 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928571    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-trusted-ca\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:44.928702 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928629    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-image-registry-private-configuration\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:44.928702 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928655    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-image-registry-private-configuration\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:44.928790 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:44.928703    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc28fe61-b116-4a8b-ad04-123d62365a02-ca-trust-extracted\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.029097 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029002    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e089d179-e583-4590-ba4c-39e7d486d707-tmp-dir\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:45.029097 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029045    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.029097 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029074    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-serving-cert\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:45.029361 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029165    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e62efb-7c65-41f3-87d4-0805eec8f7e9-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:45.029361 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.029204    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:45.029361 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.029222    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b7896669-ndwsr: secret "image-registry-tls" not found
Oct 30 12:35:45.029361 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.029289    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls podName:cc28fe61-b116-4a8b-ad04-123d62365a02 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:45.529269428 +0000 UTC m=+31.568973124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls") pod "image-registry-57b7896669-ndwsr" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02") : secret "image-registry-tls" not found
Oct 30 12:35:45.029361 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029220    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk26b\" (UniqueName: \"kubernetes.io/projected/e089d179-e583-4590-ba4c-39e7d486d707-kube-api-access-rk26b\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:45.029361 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029335    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029367    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4f13aca3-562e-406b-a154-8755b1bf7d15-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w\" (UID: \"4f13aca3-562e-406b-a154-8755b1bf7d15\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029400    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e089d179-e583-4590-ba4c-39e7d486d707-tmp-dir\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029407    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3a2f595c-6efe-4b69-81bf-074985a6ddde-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029435    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10337f54-95e9-42cd-a73f-16817fc1cb96-cert\") pod \"ingress-canary-t5j98\" (UID: \"10337f54-95e9-42cd-a73f-16817fc1cb96\") " pod="openshift-ingress-canary/ingress-canary-t5j98"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029465    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfh24\" (UniqueName: \"kubernetes.io/projected/4f13aca3-562e-406b-a154-8755b1bf7d15-kube-api-access-cfh24\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w\" (UID: \"4f13aca3-562e-406b-a154-8755b1bf7d15\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029492    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-trusted-ca\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029518    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d7qp\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-kube-api-access-2d7qp\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029544    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xftjm\" (UniqueName: \"kubernetes.io/projected/aba04aa0-2bee-47e2-9f36-8790309d6e06-kube-api-access-xftjm\") pod \"service-ca-operator-544d749c8c-x976j\" (UID: \"aba04aa0-2bee-47e2-9f36-8790309d6e06\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029566    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029589    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-ca-trust-extracted\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029646    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-installation-pull-secrets\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029675    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f13aca3-562e-406b-a154-8755b1bf7d15-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w\" (UID: \"4f13aca3-562e-406b-a154-8755b1bf7d15\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029699    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e089d179-e583-4590-ba4c-39e7d486d707-config-volume\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aba04aa0-2bee-47e2-9f36-8790309d6e06-config\") pod \"service-ca-operator-544d749c8c-x976j\" (UID: \"aba04aa0-2bee-47e2-9f36-8790309d6e06\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029747    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e62efb-7c65-41f3-87d4-0805eec8f7e9-service-ca-bundle\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:45.030518 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029775    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slh7l\" (UniqueName: \"kubernetes.io/projected/06910518-58db-450f-a5ee-521ac8f51e1a-kube-api-access-slh7l\") pod \"kas-connection-checker-685895c7c9-q74lv\" (UID: \"06910518-58db-450f-a5ee-521ac8f51e1a\") " pod="kube-system/kas-connection-checker-685895c7c9-q74lv"
Oct 30 12:35:45.031336 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.029885    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:45.529846791 +0000 UTC m=+31.569550478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : configmap references non-existent config key: service-ca.crt
Oct 30 12:35:45.031336 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.029951    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-default-certificate\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:45.031336 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030028    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4f13aca3-562e-406b-a154-8755b1bf7d15-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w\" (UID: \"4f13aca3-562e-406b-a154-8755b1bf7d15\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:45.031336 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030051    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:45.031336 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030100    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-certificates\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.031336 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030144    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-bound-sa-token\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.031336 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030177    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fbjv\" (UniqueName: \"kubernetes.io/projected/336de054-d674-4f26-81d8-980b11e27ace-kube-api-access-5fbjv\") pod \"volume-data-source-validator-58c9f5f46d-6mt4g\" (UID: \"336de054-d674-4f26-81d8-980b11e27ace\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6mt4g"
Oct 30 12:35:45.031336 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.030197    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:35:45.031336 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030222    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52p8s\" (UniqueName: \"kubernetes.io/projected/e30d5797-e034-4806-be4f-54f54698920c-kube-api-access-52p8s\") pod \"kas-connection-checker-685895c7c9-5lwxr\" (UID: \"e30d5797-e034-4806-be4f-54f54698920c\") " pod="kube-system/kas-connection-checker-685895c7c9-5lwxr"
Oct 30 12:35:45.031336 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030262    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3a2f595c-6efe-4b69-81bf-074985a6ddde-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:45.031336 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.030269    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls podName:e089d179-e583-4590-ba4c-39e7d486d707 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:45.530242223 +0000 UTC m=+31.569945899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls") pod "dns-default-2djq9" (UID: "e089d179-e583-4590-ba4c-39e7d486d707") : secret "dns-default-metrics-tls" not found
Oct 30 12:35:45.031336 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030300    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-trusted-ca\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:45.031336 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030336    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-image-registry-private-configuration\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.031336 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030364    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-image-registry-private-configuration\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.031336 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030392    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc28fe61-b116-4a8b-ad04-123d62365a02-ca-trust-extracted\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.032105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030419    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:45.032105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030445    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5nr5\" (UniqueName: \"kubernetes.io/projected/10337f54-95e9-42cd-a73f-16817fc1cb96-kube-api-access-p5nr5\") pod \"ingress-canary-t5j98\" (UID: \"10337f54-95e9-42cd-a73f-16817fc1cb96\") " pod="openshift-ingress-canary/ingress-canary-t5j98"
Oct 30 12:35:45.032105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030470    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2khhs\" (UniqueName: \"kubernetes.io/projected/2be5910f-12fa-46fe-bf9a-ddff510f4d09-kube-api-access-2khhs\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:45.032105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2v9l\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-kube-api-access-k2v9l\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.032105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030526    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzdfs\" (UniqueName: \"kubernetes.io/projected/79f0f110-a04f-4b35-8eba-c8f0469b62a1-kube-api-access-bzdfs\") pod \"kas-connection-checker-685895c7c9-q8mz6\" (UID: \"79f0f110-a04f-4b35-8eba-c8f0469b62a1\") " pod="kube-system/kas-connection-checker-685895c7c9-q8mz6"
Oct 30 12:35:45.032105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030552    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-installation-pull-secrets\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.032105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030584    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-bound-sa-token\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.032105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030622    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-certificates\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.032105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030652    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:45.032105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030679    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aba04aa0-2bee-47e2-9f36-8790309d6e06-serving-cert\") pod \"service-ca-operator-544d749c8c-x976j\" (UID: \"aba04aa0-2bee-47e2-9f36-8790309d6e06\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:45.032105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030710    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn8lm\" (UniqueName: \"kubernetes.io/projected/3a2f595c-6efe-4b69-81bf-074985a6ddde-kube-api-access-fn8lm\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:45.032105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030736    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/60e62efb-7c65-41f3-87d4-0805eec8f7e9-snapshots\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:45.032105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030739    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aba04aa0-2bee-47e2-9f36-8790309d6e06-config\") pod \"service-ca-operator-544d749c8c-x976j\" (UID: \"aba04aa0-2bee-47e2-9f36-8790309d6e06\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:45.032105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030766    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:45.032105 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.030897    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e62efb-7c65-41f3-87d4-0805eec8f7e9-service-ca-bundle\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.031015    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk2ml\" (UniqueName: \"kubernetes.io/projected/60e62efb-7c65-41f3-87d4-0805eec8f7e9-kube-api-access-xk2ml\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.031054    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-stats-auth\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.031081    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfwdk\" (UniqueName: \"kubernetes.io/projected/d81730d3-53ed-40b8-a512-348fc042f821-kube-api-access-sfwdk\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.031105    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/49837f84-8643-4643-aa92-3716a9b755d1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.031131    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b6xs\" (UniqueName: \"kubernetes.io/projected/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-kube-api-access-9b6xs\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.031158    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jknfr\" (UniqueName: \"kubernetes.io/projected/10a8a53c-d9d6-45aa-936e-48745236bb01-kube-api-access-jknfr\") pod \"network-check-source-6f78d59f99-gzkb8\" (UID: \"10a8a53c-d9d6-45aa-936e-48745236bb01\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-gzkb8"
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.031193    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-trusted-ca\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.031211    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e089d179-e583-4590-ba4c-39e7d486d707-config-volume\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.031221    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60e62efb-7c65-41f3-87d4-0805eec8f7e9-tmp\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.031254    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/60e62efb-7c65-41f3-87d4-0805eec8f7e9-serving-cert\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.031294    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.031323    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-config\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.031349    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.031375    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.031436    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert podName:49837f84-8643-4643-aa92-3716a9b755d1 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:45.531421134 +0000 UTC m=+31.571124803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-44v66" (UID: "49837f84-8643-4643-aa92-3716a9b755d1") : secret "networking-console-plugin-cert" not found
Oct 30 12:35:45.032895 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.031473    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:45.033747 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.031477    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-trusted-ca\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.033747 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.031487    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f558d755b-bj4vz: secret "image-registry-tls" not found
Oct 30 12:35:45.033747 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.031528    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls podName:bbb14793-e4b1-4912-9ce2-008cba7ce1d4 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:45.531515252 +0000 UTC m=+31.571218945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls") pod "image-registry-6f558d755b-bj4vz" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4") : secret "image-registry-tls" not found
Oct 30 12:35:45.033747 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.031483    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e62efb-7c65-41f3-87d4-0805eec8f7e9-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:45.033747 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.031868    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:35:45.033747 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.031935    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:45.531921563 +0000 UTC m=+31.571625237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : secret "router-metrics-certs-default" not found
Oct 30 12:35:45.033747 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.032162    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc28fe61-b116-4a8b-ad04-123d62365a02-ca-trust-extracted\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.033747 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.032314    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-trusted-ca\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:45.033747 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.032548    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-config\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:45.033747 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.032574    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:35:45.033747 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.032636    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls podName:2be5910f-12fa-46fe-bf9a-ddff510f4d09 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:45.532620614 +0000 UTC m=+31.572324299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-bdv44" (UID: "2be5910f-12fa-46fe-bf9a-ddff510f4d09") : secret "samples-operator-tls" not found
Oct 30 12:35:45.033747 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.032840    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/60e62efb-7c65-41f3-87d4-0805eec8f7e9-snapshots\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:45.033747 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.032952    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-ca-trust-extracted\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.033747 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.033192    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-certificates\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.033747 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.033275    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:45.034523 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.033332    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls podName:3a2f595c-6efe-4b69-81bf-074985a6ddde nodeName:}" failed. No retries permitted until 2026-10-30 12:35:45.533317109 +0000 UTC m=+31.573020803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-48t4k" (UID: "3a2f595c-6efe-4b69-81bf-074985a6ddde") : secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:45.034523 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.033523    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-certificates\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.034523 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.034116    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:45.034688 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.034568    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f13aca3-562e-406b-a154-8755b1bf7d15-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w\" (UID: \"4f13aca3-562e-406b-a154-8755b1bf7d15\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:45.034944 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.034918    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-trusted-ca\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.035114 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.035051    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60e62efb-7c65-41f3-87d4-0805eec8f7e9-tmp\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:45.035388 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.035366    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-default-certificate\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:45.035557 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.035538    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/49837f84-8643-4643-aa92-3716a9b755d1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:45.035740 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.035704    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-image-registry-private-configuration\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.036031 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.036007    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aba04aa0-2bee-47e2-9f36-8790309d6e06-serving-cert\") pod \"service-ca-operator-544d749c8c-x976j\" (UID: \"aba04aa0-2bee-47e2-9f36-8790309d6e06\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:45.036622 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.036569    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-serving-cert\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:45.036873 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.036839    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-stats-auth\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:45.037413 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.037355    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-installation-pull-secrets\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.037413 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.037375    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10337f54-95e9-42cd-a73f-16817fc1cb96-cert\") pod \"ingress-canary-t5j98\" (UID: \"10337f54-95e9-42cd-a73f-16817fc1cb96\") " pod="openshift-ingress-canary/ingress-canary-t5j98"
Oct 30 12:35:45.037555 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.037439    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/60e62efb-7c65-41f3-87d4-0805eec8f7e9-serving-cert\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:45.037673 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.037656    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-installation-pull-secrets\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.037725 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.037676    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-image-registry-private-configuration\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.049920 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.049491    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slh7l\" (UniqueName: \"kubernetes.io/projected/06910518-58db-450f-a5ee-521ac8f51e1a-kube-api-access-slh7l\") pod \"kas-connection-checker-685895c7c9-q74lv\" (UID: \"06910518-58db-450f-a5ee-521ac8f51e1a\") " pod="kube-system/kas-connection-checker-685895c7c9-q74lv"
Oct 30 12:35:45.049920 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.049889    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfh24\" (UniqueName: \"kubernetes.io/projected/4f13aca3-562e-406b-a154-8755b1bf7d15-kube-api-access-cfh24\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w\" (UID: \"4f13aca3-562e-406b-a154-8755b1bf7d15\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:45.052776 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.051963    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2v9l\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-kube-api-access-k2v9l\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.052776 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.052338    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-bound-sa-token\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.052776 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.052731    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52p8s\" (UniqueName: \"kubernetes.io/projected/e30d5797-e034-4806-be4f-54f54698920c-kube-api-access-52p8s\") pod \"kas-connection-checker-685895c7c9-5lwxr\" (UID: \"e30d5797-e034-4806-be4f-54f54698920c\") " pod="kube-system/kas-connection-checker-685895c7c9-5lwxr"
Oct 30 12:35:45.052993 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.052816    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn8lm\" (UniqueName: \"kubernetes.io/projected/3a2f595c-6efe-4b69-81bf-074985a6ddde-kube-api-access-fn8lm\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:45.053400 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.053378    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xftjm\" (UniqueName: \"kubernetes.io/projected/aba04aa0-2bee-47e2-9f36-8790309d6e06-kube-api-access-xftjm\") pod \"service-ca-operator-544d749c8c-x976j\" (UID: \"aba04aa0-2bee-47e2-9f36-8790309d6e06\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:45.055269 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.055243    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5nr5\" (UniqueName: \"kubernetes.io/projected/10337f54-95e9-42cd-a73f-16817fc1cb96-kube-api-access-p5nr5\") pod \"ingress-canary-t5j98\" (UID: \"10337f54-95e9-42cd-a73f-16817fc1cb96\") " pod="openshift-ingress-canary/ingress-canary-t5j98"
Oct 30 12:35:45.055870 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.055548    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jknfr\" (UniqueName: \"kubernetes.io/projected/10a8a53c-d9d6-45aa-936e-48745236bb01-kube-api-access-jknfr\") pod \"network-check-source-6f78d59f99-gzkb8\" (UID: \"10a8a53c-d9d6-45aa-936e-48745236bb01\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-gzkb8"
Oct 30 12:35:45.055870 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.055751    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfwdk\" (UniqueName: \"kubernetes.io/projected/d81730d3-53ed-40b8-a512-348fc042f821-kube-api-access-sfwdk\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:45.055870 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.055827    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk26b\" (UniqueName: \"kubernetes.io/projected/e089d179-e583-4590-ba4c-39e7d486d707-kube-api-access-rk26b\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:45.056061 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.055875    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b6xs\" (UniqueName: \"kubernetes.io/projected/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-kube-api-access-9b6xs\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:45.056061 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.055896    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk2ml\" (UniqueName: \"kubernetes.io/projected/60e62efb-7c65-41f3-87d4-0805eec8f7e9-kube-api-access-xk2ml\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:45.056158 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.056068    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fbjv\" (UniqueName: \"kubernetes.io/projected/336de054-d674-4f26-81d8-980b11e27ace-kube-api-access-5fbjv\") pod \"volume-data-source-validator-58c9f5f46d-6mt4g\" (UID: \"336de054-d674-4f26-81d8-980b11e27ace\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6mt4g"
Oct 30 12:35:45.056158 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.056127    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2khhs\" (UniqueName: \"kubernetes.io/projected/2be5910f-12fa-46fe-bf9a-ddff510f4d09-kube-api-access-2khhs\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:45.056227 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.056160    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-bound-sa-token\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.056463 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.056446    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzdfs\" (UniqueName: \"kubernetes.io/projected/79f0f110-a04f-4b35-8eba-c8f0469b62a1-kube-api-access-bzdfs\") pod \"kas-connection-checker-685895c7c9-q8mz6\" (UID: \"79f0f110-a04f-4b35-8eba-c8f0469b62a1\") " pod="kube-system/kas-connection-checker-685895c7c9-q8mz6"
Oct 30 12:35:45.057072 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.057057    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d7qp\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-kube-api-access-2d7qp\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.090447 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.090423    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-5lwxr"
Oct 30 12:35:45.123459 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.123438    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:45.144155 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.144136    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-q74lv"
Oct 30 12:35:45.157006 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.156981    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-q8mz6"
Oct 30 12:35:45.168645 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.168622    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:45.175023 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.175000    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:45.179849 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.179821    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:45.186316 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.186275    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6mt4g"
Oct 30 12:35:45.211827 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.211718    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-gzkb8"
Oct 30 12:35:45.227327 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.227244    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t5j98"
Oct 30 12:35:45.433263 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.433235    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-5lwxr"]
Oct 30 12:35:45.482252 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.482180    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-q74lv"]
Oct 30 12:35:45.486076 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.486002    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"]
Oct 30 12:35:45.487176 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:45.486425    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06910518_58db_450f_a5ee_521ac8f51e1a.slice/crio-966dcd2556e1ec1346a605685f267af7369016b91467bf9b088d1fd25d4ad694 WatchSource:0}: Error finding container 966dcd2556e1ec1346a605685f267af7369016b91467bf9b088d1fd25d4ad694: Status 404 returned error can't find the container with id 966dcd2556e1ec1346a605685f267af7369016b91467bf9b088d1fd25d4ad694
Oct 30 12:35:45.500297 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.500276    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-q8mz6"]
Oct 30 12:35:45.518475 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.518418    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6mt4g"]
Oct 30 12:35:45.523853 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:45.523827    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336de054_d674_4f26_81d8_980b11e27ace.slice/crio-69652a499a9fef069224a36d8831b8831c15ee6dde86a929b4e48599cf0044f2 WatchSource:0}: Error finding container 69652a499a9fef069224a36d8831b8831c15ee6dde86a929b4e48599cf0044f2: Status 404 returned error can't find the container with id 69652a499a9fef069224a36d8831b8831c15ee6dde86a929b4e48599cf0044f2
Oct 30 12:35:45.542780 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.542748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:45.542879 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.542828    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:45.543050 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.542886    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:45.543050 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.542954    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:45.543050 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.542990    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:45.543050 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.543044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:45.543235 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.543112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:45.543235 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.543191    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.543381    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.543475    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls podName:2be5910f-12fa-46fe-bf9a-ddff510f4d09 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:46.543454265 +0000 UTC m=+32.583157949 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-bdv44" (UID: "2be5910f-12fa-46fe-bf9a-ddff510f4d09") : secret "samples-operator-tls" not found
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.543796    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.543866    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls podName:3a2f595c-6efe-4b69-81bf-074985a6ddde nodeName:}" failed. No retries permitted until 2026-10-30 12:35:46.543833371 +0000 UTC m=+32.583537057 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-48t4k" (UID: "3a2f595c-6efe-4b69-81bf-074985a6ddde") : secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.543922    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.543942    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f558d755b-bj4vz: secret "image-registry-tls" not found
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.543970    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.543983    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b7896669-ndwsr: secret "image-registry-tls" not found
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.543993    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls podName:bbb14793-e4b1-4912-9ce2-008cba7ce1d4 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:46.54397682 +0000 UTC m=+32.583680493 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls") pod "image-registry-6f558d755b-bj4vz" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4") : secret "image-registry-tls" not found
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.544035    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls podName:cc28fe61-b116-4a8b-ad04-123d62365a02 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:46.544022901 +0000 UTC m=+32.583726570 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls") pod "image-registry-57b7896669-ndwsr" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02") : secret "image-registry-tls" not found
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.544106    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.544121    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:46.544111029 +0000 UTC m=+32.583814733 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : configmap references non-existent config key: service-ca.crt
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.544147    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:46.544135053 +0000 UTC m=+32.583838725 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : secret "router-metrics-certs-default" not found
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.544209    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.544263    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls podName:e089d179-e583-4590-ba4c-39e7d486d707 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:46.54423466 +0000 UTC m=+32.583938350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls") pod "dns-default-2djq9" (UID: "e089d179-e583-4590-ba4c-39e7d486d707") : secret "dns-default-metrics-tls" not found
Oct 30 12:35:45.544682 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.544209    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:35:45.548700 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:45.544306    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert podName:49837f84-8643-4643-aa92-3716a9b755d1 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:46.54429698 +0000 UTC m=+32.584000652 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-44v66" (UID: "49837f84-8643-4643-aa92-3716a9b755d1") : secret "networking-console-plugin-cert" not found
Oct 30 12:35:45.558582 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.558516    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-m6xzg"]
Oct 30 12:35:45.565079 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:45.565058    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e62efb_7c65_41f3_87d4_0805eec8f7e9.slice/crio-e73ede3436e8237d52c9d329de9e46c44e189332e305ccdad50cf1eb4df1dbdd WatchSource:0}: Error finding container e73ede3436e8237d52c9d329de9e46c44e189332e305ccdad50cf1eb4df1dbdd: Status 404 returned error can't find the container with id e73ede3436e8237d52c9d329de9e46c44e189332e305ccdad50cf1eb4df1dbdd
Oct 30 12:35:45.585215 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.585190    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t5j98"]
Oct 30 12:35:45.589959 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.589935    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"]
Oct 30 12:35:45.593366 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:45.593316    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f13aca3_562e_406b_a154_8755b1bf7d15.slice/crio-ab1eba861addee696d5063000eae19b0fb3f0aa5c112d7ab33a2732fc5767e8e WatchSource:0}: Error finding container ab1eba861addee696d5063000eae19b0fb3f0aa5c112d7ab33a2732fc5767e8e: Status 404 returned error can't find the container with id ab1eba861addee696d5063000eae19b0fb3f0aa5c112d7ab33a2732fc5767e8e
Oct 30 12:35:45.594139 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:45.594121    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10337f54_95e9_42cd_a73f_16817fc1cb96.slice/crio-dc2f830ae39d642b0c9b00c36a1f943093a33bb54659bc00bb35c1fd34a304e2 WatchSource:0}: Error finding container dc2f830ae39d642b0c9b00c36a1f943093a33bb54659bc00bb35c1fd34a304e2: Status 404 returned error can't find the container with id dc2f830ae39d642b0c9b00c36a1f943093a33bb54659bc00bb35c1fd34a304e2
Oct 30 12:35:45.598125 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.598058    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"]
Oct 30 12:35:45.605551 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:45.605531    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaba04aa0_2bee_47e2_9f36_8790309d6e06.slice/crio-4abb3ff147c2ca169b46c138813140b41dbe52d3eab89d6b4a2c31371d0197d7 WatchSource:0}: Error finding container 4abb3ff147c2ca169b46c138813140b41dbe52d3eab89d6b4a2c31371d0197d7: Status 404 returned error can't find the container with id 4abb3ff147c2ca169b46c138813140b41dbe52d3eab89d6b4a2c31371d0197d7
Oct 30 12:35:45.608320 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.608297    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-gzkb8"]
Oct 30 12:35:45.613408 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:45.613365    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a8a53c_d9d6_45aa_936e_48745236bb01.slice/crio-4d6911b025d9e88987f3642ba349b3c4e1559445b1f9f5408d99e1a9ed1e51a9 WatchSource:0}: Error finding container 4d6911b025d9e88987f3642ba349b3c4e1559445b1f9f5408d99e1a9ed1e51a9: Status 404 returned error can't find the container with id 4d6911b025d9e88987f3642ba349b3c4e1559445b1f9f5408d99e1a9ed1e51a9
Oct 30 12:35:45.754838 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.754808    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-gzkb8" event={"ID":"10a8a53c-d9d6-45aa-936e-48745236bb01","Type":"ContainerStarted","Data":"4d6911b025d9e88987f3642ba349b3c4e1559445b1f9f5408d99e1a9ed1e51a9"}
Oct 30 12:35:45.756012 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.755987    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t5j98" event={"ID":"10337f54-95e9-42cd-a73f-16817fc1cb96","Type":"ContainerStarted","Data":"dc2f830ae39d642b0c9b00c36a1f943093a33bb54659bc00bb35c1fd34a304e2"}
Oct 30 12:35:45.756978 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.756962    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j" event={"ID":"aba04aa0-2bee-47e2-9f36-8790309d6e06","Type":"ContainerStarted","Data":"4abb3ff147c2ca169b46c138813140b41dbe52d3eab89d6b4a2c31371d0197d7"}
Oct 30 12:35:45.757962 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.757940    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w" event={"ID":"4f13aca3-562e-406b-a154-8755b1bf7d15","Type":"ContainerStarted","Data":"ab1eba861addee696d5063000eae19b0fb3f0aa5c112d7ab33a2732fc5767e8e"}
Oct 30 12:35:45.759030 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.759012    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6mt4g" event={"ID":"336de054-d674-4f26-81d8-980b11e27ace","Type":"ContainerStarted","Data":"69652a499a9fef069224a36d8831b8831c15ee6dde86a929b4e48599cf0044f2"}
Oct 30 12:35:45.760008 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.759978    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-m6xzg" event={"ID":"60e62efb-7c65-41f3-87d4-0805eec8f7e9","Type":"ContainerStarted","Data":"e73ede3436e8237d52c9d329de9e46c44e189332e305ccdad50cf1eb4df1dbdd"}
Oct 30 12:35:45.761214 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.761189    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-5lwxr" event={"ID":"e30d5797-e034-4806-be4f-54f54698920c","Type":"ContainerStarted","Data":"d2407134541c0aaf9585d3687b7745dbb2fce5aafa580104fcd39b280151ce95"}
Oct 30 12:35:45.761307 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.761225    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-5lwxr" event={"ID":"e30d5797-e034-4806-be4f-54f54698920c","Type":"ContainerStarted","Data":"3b7b2649a9fbcd99e7bf75a6aac80cf8db43aec6c649374e9a5603007884136f"}
Oct 30 12:35:45.762701 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.762682    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-q8mz6" event={"ID":"79f0f110-a04f-4b35-8eba-c8f0469b62a1","Type":"ContainerStarted","Data":"ea49c5611a9537a3a31f078dfa474c4d03e7a70241b8389f774c15fb5287256b"}
Oct 30 12:35:45.762783 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.762714    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-q8mz6" event={"ID":"79f0f110-a04f-4b35-8eba-c8f0469b62a1","Type":"ContainerStarted","Data":"918f40e9b236196597ce1d674a12bd818bd975806363262c6f485da82c1a5346"}
Oct 30 12:35:45.764108 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.764089    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-q74lv" event={"ID":"06910518-58db-450f-a5ee-521ac8f51e1a","Type":"ContainerStarted","Data":"ef960db7f326308f4c1326566938aaa9d7c946f3e895e9a035be636301b786f9"}
Oct 30 12:35:45.764200 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.764143    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-q74lv" event={"ID":"06910518-58db-450f-a5ee-521ac8f51e1a","Type":"ContainerStarted","Data":"966dcd2556e1ec1346a605685f267af7369016b91467bf9b088d1fd25d4ad694"}
Oct 30 12:35:45.765184 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.765165    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8" event={"ID":"b80ffbfe-445f-431c-b8f4-be9fd2c5d094","Type":"ContainerStarted","Data":"57296fb69fdd6aa9c6628c3621c1b66d19e6d3956c607a31461b6910e5294233"}
Oct 30 12:35:45.801792 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.801718    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-5lwxr" podStartSLOduration=412.801705711 podStartE2EDuration="6m52.801705711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:45.800698575 +0000 UTC m=+31.840402269" watchObservedRunningTime="2026-10-30 12:35:45.801705711 +0000 UTC m=+31.841409402"
Oct 30 12:35:45.837065 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.837018    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-q8mz6" podStartSLOduration=412.837003584 podStartE2EDuration="6m52.837003584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:45.836411759 +0000 UTC m=+31.876115462" watchObservedRunningTime="2026-10-30 12:35:45.837003584 +0000 UTC m=+31.876707274"
Oct 30 12:35:45.872250 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:45.872215    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-q74lv" podStartSLOduration=412.872204225 podStartE2EDuration="6m52.872204225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:45.872030776 +0000 UTC m=+31.911734466" watchObservedRunningTime="2026-10-30 12:35:45.872204225 +0000 UTC m=+31.911907915"
Oct 30 12:35:46.513935 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.513887    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:46.513935 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.513902    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:46.514157 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.514107    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:46.532303 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.532205    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-bg2k7\""
Oct 30 12:35:46.532303 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.532228    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 12:35:46.532303 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.532215    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 12:35:46.532303 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.532273    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-t6s7j\""
Oct 30 12:35:46.534567 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.534517    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-54vc4\""
Oct 30 12:35:46.555702 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.554246    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:46.555702 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.554302    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:46.555702 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.554342    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:46.555702 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.554378    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:46.555702 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.554424    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:46.555702 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.554456    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:46.555702 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.554490    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:46.555702 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:46.554535    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:46.555702 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.554682    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:35:46.555702 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.554734    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls podName:e089d179-e583-4590-ba4c-39e7d486d707 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:48.554716443 +0000 UTC m=+34.594420115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls") pod "dns-default-2djq9" (UID: "e089d179-e583-4590-ba4c-39e7d486d707") : secret "dns-default-metrics-tls" not found
Oct 30 12:35:46.555702 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.555116    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:35:46.555702 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.555165    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls podName:2be5910f-12fa-46fe-bf9a-ddff510f4d09 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:48.555149321 +0000 UTC m=+34.594853006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-bdv44" (UID: "2be5910f-12fa-46fe-bf9a-ddff510f4d09") : secret "samples-operator-tls" not found
Oct 30 12:35:46.555702 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.555224    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:46.555702 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.555256    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls podName:3a2f595c-6efe-4b69-81bf-074985a6ddde nodeName:}" failed. No retries permitted until 2026-10-30 12:35:48.555245432 +0000 UTC m=+34.594949102 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-48t4k" (UID: "3a2f595c-6efe-4b69-81bf-074985a6ddde") : secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:46.555702 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.555305    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:35:46.556477 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.555333    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:48.555323305 +0000 UTC m=+34.595026979 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : secret "router-metrics-certs-default" not found
Oct 30 12:35:46.556477 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.555381    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:35:46.556477 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.555416    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert podName:49837f84-8643-4643-aa92-3716a9b755d1 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:48.555402029 +0000 UTC m=+34.595105697 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-44v66" (UID: "49837f84-8643-4643-aa92-3716a9b755d1") : secret "networking-console-plugin-cert" not found
Oct 30 12:35:46.556477 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.555471    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:46.556477 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.555481    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f558d755b-bj4vz: secret "image-registry-tls" not found
Oct 30 12:35:46.556477 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.555509    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls podName:bbb14793-e4b1-4912-9ce2-008cba7ce1d4 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:48.555499885 +0000 UTC m=+34.595203558 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls") pod "image-registry-6f558d755b-bj4vz" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4") : secret "image-registry-tls" not found
Oct 30 12:35:46.556477 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.555557    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:46.556477 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.555566    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b7896669-ndwsr: secret "image-registry-tls" not found
Oct 30 12:35:46.556477 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.555593    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls podName:cc28fe61-b116-4a8b-ad04-123d62365a02 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:48.555583809 +0000 UTC m=+34.595287482 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls") pod "image-registry-57b7896669-ndwsr" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02") : secret "image-registry-tls" not found
Oct 30 12:35:46.556477 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:46.555670    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:48.555658747 +0000 UTC m=+34.595362418 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : configmap references non-existent config key: service-ca.crt
Oct 30 12:35:48.274948 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.274902    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:48.275474 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.275013    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:48.275742 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.275718    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 12:35:48.275844 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.275782    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs podName:2758dbad-1f8a-4503-9dc2-6181518daeaf nodeName:}" failed. No retries permitted until 2026-10-30 12:36:20.275767397 +0000 UTC m=+66.315471066 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs") pod "network-metrics-daemon-xdcfv" (UID: "2758dbad-1f8a-4503-9dc2-6181518daeaf") : secret "metrics-daemon-secret" not found
Oct 30 12:35:48.278230 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.278209    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:48.367491 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.367280    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:48.376115 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.376089    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lshc4\" (UniqueName: \"kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4\") pod \"network-check-target-29mpd\" (UID: \"a7e45249-4b17-415f-966d-af5321f182d5\") " pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:48.381351 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.381300    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lshc4\" (UniqueName: \"kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4\") pod \"network-check-target-29mpd\" (UID: \"a7e45249-4b17-415f-966d-af5321f182d5\") " pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.590085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.590142    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.590190    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.590233    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.590279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.590316    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.590341    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.590380    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.590516    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.590530    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f558d755b-bj4vz: secret "image-registry-tls" not found
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.590587    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls podName:bbb14793-e4b1-4912-9ce2-008cba7ce1d4 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:52.590569428 +0000 UTC m=+38.630273111 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls") pod "image-registry-6f558d755b-bj4vz" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4") : secret "image-registry-tls" not found
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.590761    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.590814    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls podName:2be5910f-12fa-46fe-bf9a-ddff510f4d09 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:52.590797887 +0000 UTC m=+38.630501571 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-bdv44" (UID: "2be5910f-12fa-46fe-bf9a-ddff510f4d09") : secret "samples-operator-tls" not found
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.590858    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.590865    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b7896669-ndwsr: secret "image-registry-tls" not found
Oct 30 12:35:48.591243 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.590894    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls podName:cc28fe61-b116-4a8b-ad04-123d62365a02 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:52.590883819 +0000 UTC m=+38.630587503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls") pod "image-registry-57b7896669-ndwsr" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02") : secret "image-registry-tls" not found
Oct 30 12:35:48.592123 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.590946    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:52.590939271 +0000 UTC m=+38.630642952 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : configmap references non-existent config key: service-ca.crt
Oct 30 12:35:48.592123 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.591001    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:35:48.592123 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.591031    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls podName:e089d179-e583-4590-ba4c-39e7d486d707 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:52.591021235 +0000 UTC m=+38.630724919 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls") pod "dns-default-2djq9" (UID: "e089d179-e583-4590-ba4c-39e7d486d707") : secret "dns-default-metrics-tls" not found
Oct 30 12:35:48.592123 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.591084    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:35:48.592123 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.591116    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:52.591106004 +0000 UTC m=+38.630809687 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : secret "router-metrics-certs-default" not found
Oct 30 12:35:48.592123 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.591155    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:48.592123 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.591177    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls podName:3a2f595c-6efe-4b69-81bf-074985a6ddde nodeName:}" failed. No retries permitted until 2026-10-30 12:35:52.591170457 +0000 UTC m=+38.630874138 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-48t4k" (UID: "3a2f595c-6efe-4b69-81bf-074985a6ddde") : secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:48.592123 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.591208    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:35:48.592123 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:48.591227    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert podName:49837f84-8643-4643-aa92-3716a9b755d1 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:52.591221184 +0000 UTC m=+38.630924866 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-44v66" (UID: "49837f84-8643-4643-aa92-3716a9b755d1") : secret "networking-console-plugin-cert" not found
Oct 30 12:35:48.648378 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.648302    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:48.711281 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.711213    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ntzbv"]
Oct 30 12:35:48.781872 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.781838    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6mt4g" event={"ID":"336de054-d674-4f26-81d8-980b11e27ace","Type":"ContainerStarted","Data":"1f6c8874f1abe82ed72b6379be25fad194f5fe1611279ba95b6227650a0ca26b"}
Oct 30 12:35:48.784093 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.784063    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ntzbv" event={"ID":"15bb3274-f845-4851-90ae-f4ab08e685c8","Type":"ContainerStarted","Data":"c88512809f317fb4be946430d40291f8d55c07f7231005bd8d88bd9496953990"}
Oct 30 12:35:48.818949 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.818889    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6mt4g" podStartSLOduration=386.636230538 podStartE2EDuration="6m28.81887212s" totalImagesPullingTime="2.182641582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:29:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:48.817985116 +0000 UTC m=+34.857688834" watchObservedRunningTime="2026-10-30 12:35:48.81887212 +0000 UTC m=+34.858575816"
Oct 30 12:35:48.966591 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:48.966504    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-29mpd"]
Oct 30 12:35:48.976815 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:48.976784    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e45249_4b17_415f_966d_af5321f182d5.slice/crio-1af18b9b9ea3cf77550be3bc00adbc0fb260e197d40cdf8e54ab968dc6323fff WatchSource:0}: Error finding container 1af18b9b9ea3cf77550be3bc00adbc0fb260e197d40cdf8e54ab968dc6323fff: Status 404 returned error can't find the container with id 1af18b9b9ea3cf77550be3bc00adbc0fb260e197d40cdf8e54ab968dc6323fff
Oct 30 12:35:49.792475 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:49.792430    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8" event={"ID":"b80ffbfe-445f-431c-b8f4-be9fd2c5d094","Type":"ContainerStarted","Data":"910db2d7e9eab8901f30c27d7461eea3efe92ac591446747dd74f9f532790a61"}
Oct 30 12:35:49.793840 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:49.793497    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:49.794946 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:49.794911    2581 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-8gvm8 container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.8:8443/readyz\": dial tcp 10.132.0.8:8443: connect: connection refused" start-of-body=
Oct 30 12:35:49.795039 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:49.794975    2581 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8" podUID="b80ffbfe-445f-431c-b8f4-be9fd2c5d094" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.8:8443/readyz\": dial tcp 10.132.0.8:8443: connect: connection refused"
Oct 30 12:35:49.797360 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:49.797332    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-29mpd" event={"ID":"a7e45249-4b17-415f-966d-af5321f182d5","Type":"ContainerStarted","Data":"1af18b9b9ea3cf77550be3bc00adbc0fb260e197d40cdf8e54ab968dc6323fff"}
Oct 30 12:35:50.808351 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:50.808312    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/0.log"
Oct 30 12:35:50.808966 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:50.808383    2581 generic.go:356] "Generic (PLEG): container finished" podID="b80ffbfe-445f-431c-b8f4-be9fd2c5d094" containerID="910db2d7e9eab8901f30c27d7461eea3efe92ac591446747dd74f9f532790a61" exitCode=255
Oct 30 12:35:50.808966 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:50.808594    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8" event={"ID":"b80ffbfe-445f-431c-b8f4-be9fd2c5d094","Type":"ContainerDied","Data":"910db2d7e9eab8901f30c27d7461eea3efe92ac591446747dd74f9f532790a61"}
Oct 30 12:35:50.809085 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:50.809030    2581 scope.go:122] "RemoveContainer" containerID="910db2d7e9eab8901f30c27d7461eea3efe92ac591446747dd74f9f532790a61"
Oct 30 12:35:50.815263 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:50.815232    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j" event={"ID":"aba04aa0-2bee-47e2-9f36-8790309d6e06","Type":"ContainerStarted","Data":"5a4a3465b092e40effab90d846efc344d30c5628a10a84d026ec220db03afbd3"}
Oct 30 12:35:50.818328 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:50.818289    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w" event={"ID":"4f13aca3-562e-406b-a154-8755b1bf7d15","Type":"ContainerStarted","Data":"12b8abefe8eed3bb0629ff0eb5eaf5b1bdeb68462c185ed1b373600bd9b8a13b"}
Oct 30 12:35:50.823521 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:50.823494    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-m6xzg" event={"ID":"60e62efb-7c65-41f3-87d4-0805eec8f7e9","Type":"ContainerStarted","Data":"fe6b7ed38e8f251060a656eb1b02f153d53fa334a8866792f862087df17121e4"}
Oct 30 12:35:50.889155 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:50.889111    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-m6xzg" podStartSLOduration=386.387790781 podStartE2EDuration="6m30.889096271s" totalImagesPullingTime="4.50130549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:29:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:50.887827719 +0000 UTC m=+36.927531411" watchObservedRunningTime="2026-10-30 12:35:50.889096271 +0000 UTC m=+36.928799961"
Oct 30 12:35:50.931654 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:50.928079    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j" podStartSLOduration=385.880288068 podStartE2EDuration="6m30.928062681s" totalImagesPullingTime="5.047774613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:29:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:50.926071132 +0000 UTC m=+36.965774823" watchObservedRunningTime="2026-10-30 12:35:50.928062681 +0000 UTC m=+36.967766373"
Oct 30 12:35:50.973857 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:50.970694    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w" podStartSLOduration=385.894915158 podStartE2EDuration="6m30.970677969s" totalImagesPullingTime="5.075762811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:29:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:50.970345143 +0000 UTC m=+37.010048835" watchObservedRunningTime="2026-10-30 12:35:50.970677969 +0000 UTC m=+37.010381661"
Oct 30 12:35:51.038625 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.037594    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lw4gw"]
Oct 30 12:35:51.053384 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.053361    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.065630 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.060247    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lw4gw"]
Oct 30 12:35:51.075102 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.075081    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 12:35:51.075240 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.075226    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t86w7\""
Oct 30 12:35:51.075314 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.075302    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 12:35:51.125348 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.125323    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.125492 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.125402    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ddnv\" (UniqueName: \"kubernetes.io/projected/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-api-access-4ddnv\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.125492 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.125431    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.125492 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.125476    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b4171e4a-4d6e-4901-983a-600ab76ada45-crio-socket\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.125492 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.125490    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4171e4a-4d6e-4901-983a-600ab76ada45-data-volume\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.228094 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.228059    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.228209 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.228148    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ddnv\" (UniqueName: \"kubernetes.io/projected/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-api-access-4ddnv\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.228209 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.228186    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.228315 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.228250    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b4171e4a-4d6e-4901-983a-600ab76ada45-crio-socket\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.228315 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.228279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4171e4a-4d6e-4901-983a-600ab76ada45-data-volume\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.228760 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.228734    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4171e4a-4d6e-4901-983a-600ab76ada45-data-volume\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.228857 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:51.228826    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 12:35:51.228913 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:51.228884    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm podName:b4171e4a-4d6e-4901-983a-600ab76ada45 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:51.728865563 +0000 UTC m=+37.768569233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lw4gw" (UID: "b4171e4a-4d6e-4901-983a-600ab76ada45") : configmap "kube-rbac-proxy" not found
Oct 30 12:35:51.229300 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:51.229279    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:35:51.229386 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:51.229335    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls podName:b4171e4a-4d6e-4901-983a-600ab76ada45 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:51.729319079 +0000 UTC m=+37.769022748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lw4gw" (UID: "b4171e4a-4d6e-4901-983a-600ab76ada45") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:35:51.229503 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.229486    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b4171e4a-4d6e-4901-983a-600ab76ada45-crio-socket\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.256953 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.256915    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ddnv\" (UniqueName: \"kubernetes.io/projected/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-api-access-4ddnv\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.733243 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.733202    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.733418 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.733393    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:51.733558 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:51.733542    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 12:35:51.733649 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:51.733637    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm podName:b4171e4a-4d6e-4901-983a-600ab76ada45 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:52.733616979 +0000 UTC m=+38.773320662 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lw4gw" (UID: "b4171e4a-4d6e-4901-983a-600ab76ada45") : configmap "kube-rbac-proxy" not found
Oct 30 12:35:51.734093 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:51.734062    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:35:51.734222 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:51.734126    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls podName:b4171e4a-4d6e-4901-983a-600ab76ada45 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:52.734112243 +0000 UTC m=+38.773815915 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lw4gw" (UID: "b4171e4a-4d6e-4901-983a-600ab76ada45") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:35:51.835939 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.835909    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/1.log"
Oct 30 12:35:51.836420 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.836400    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/0.log"
Oct 30 12:35:51.836494 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.836437    2581 generic.go:356] "Generic (PLEG): container finished" podID="b80ffbfe-445f-431c-b8f4-be9fd2c5d094" containerID="962e40dd4a1466cf55a7d8df3804586fbec551f4fb8919a94013f65991c6296b" exitCode=255
Oct 30 12:35:51.837361 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.836755    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8" event={"ID":"b80ffbfe-445f-431c-b8f4-be9fd2c5d094","Type":"ContainerDied","Data":"962e40dd4a1466cf55a7d8df3804586fbec551f4fb8919a94013f65991c6296b"}
Oct 30 12:35:51.837361 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.836823    2581 scope.go:122] "RemoveContainer" containerID="910db2d7e9eab8901f30c27d7461eea3efe92ac591446747dd74f9f532790a61"
Oct 30 12:35:51.837361 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.836932    2581 scope.go:122] "RemoveContainer" containerID="962e40dd4a1466cf55a7d8df3804586fbec551f4fb8919a94013f65991c6296b"
Oct 30 12:35:51.837361 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:51.837144    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-8gvm8_openshift-console-operator(b80ffbfe-445f-431c-b8f4-be9fd2c5d094)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8" podUID="b80ffbfe-445f-431c-b8f4-be9fd2c5d094"
Oct 30 12:35:51.843113 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.842808    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t5j98" event={"ID":"10337f54-95e9-42cd-a73f-16817fc1cb96","Type":"ContainerStarted","Data":"11b58374cbdaa645d3b9612a78f3f0e944c8ee70c3153d3af2a2dc742684a8b7"}
Oct 30 12:35:51.915428 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:51.915375    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-t5j98" podStartSLOduration=2.348660221 podStartE2EDuration="7.91536143s" totalImagesPullingTime="5.566701209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:35:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:51.913689221 +0000 UTC m=+37.953392911" watchObservedRunningTime="2026-10-30 12:35:51.91536143 +0000 UTC m=+37.955065121"
Oct 30 12:35:52.143628 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.142653    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-tjlfm"]
Oct 30 12:35:52.162143 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.162122    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tjlfm"
Oct 30 12:35:52.171396 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.171372    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-tjlfm"]
Oct 30 12:35:52.176176 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.176137    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:52.176659 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.176635    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 12:35:52.176781 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.176762    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2mbkm\""
Oct 30 12:35:52.242012 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.241984    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h54zx\" (UniqueName: \"kubernetes.io/projected/eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d-kube-api-access-h54zx\") pod \"migrator-66df5d496b-tjlfm\" (UID: \"eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tjlfm"
Oct 30 12:35:52.343210 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.343127    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54zx\" (UniqueName: \"kubernetes.io/projected/eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d-kube-api-access-h54zx\") pod \"migrator-66df5d496b-tjlfm\" (UID: \"eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tjlfm"
Oct 30 12:35:52.361805 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.361749    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h54zx\" (UniqueName: \"kubernetes.io/projected/eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d-kube-api-access-h54zx\") pod \"migrator-66df5d496b-tjlfm\" (UID: \"eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tjlfm"
Oct 30 12:35:52.488373 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.488346    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tjlfm"
Oct 30 12:35:52.646286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.646214    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:52.646286 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.646262    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:52.646455 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.646298    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:52.646455 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.646319    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:52.646455 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.646361    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:52.646455 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.646386    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:52.646455 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646402    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:35:52.646455 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.646421    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:52.646455 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.646455    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:52.646696 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646479    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.646458908 +0000 UTC m=+46.686162594 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : secret "router-metrics-certs-default" not found
Oct 30 12:35:52.646696 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646522    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:52.646696 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646561    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls podName:3a2f595c-6efe-4b69-81bf-074985a6ddde nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.646549768 +0000 UTC m=+46.686253441 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-48t4k" (UID: "3a2f595c-6efe-4b69-81bf-074985a6ddde") : secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:52.646696 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646625    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:35:52.646696 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646650    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert podName:49837f84-8643-4643-aa92-3716a9b755d1 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.646642962 +0000 UTC m=+46.686346631 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-44v66" (UID: "49837f84-8643-4643-aa92-3716a9b755d1") : secret "networking-console-plugin-cert" not found
Oct 30 12:35:52.646696 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646690    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:52.646696 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646697    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f558d755b-bj4vz: secret "image-registry-tls" not found
Oct 30 12:35:52.646908 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646717    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls podName:bbb14793-e4b1-4912-9ce2-008cba7ce1d4 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.646710869 +0000 UTC m=+46.686414538 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls") pod "image-registry-6f558d755b-bj4vz" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4") : secret "image-registry-tls" not found
Oct 30 12:35:52.646908 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646750    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:52.646908 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646756    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b7896669-ndwsr: secret "image-registry-tls" not found
Oct 30 12:35:52.646908 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646775    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls podName:cc28fe61-b116-4a8b-ad04-123d62365a02 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.646768901 +0000 UTC m=+46.686472574 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls") pod "image-registry-57b7896669-ndwsr" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02") : secret "image-registry-tls" not found
Oct 30 12:35:52.646908 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646826    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.646820217 +0000 UTC m=+46.686523886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : configmap references non-existent config key: service-ca.crt
Oct 30 12:35:52.646908 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646862    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:35:52.646908 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646879    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls podName:e089d179-e583-4590-ba4c-39e7d486d707 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.646873988 +0000 UTC m=+46.686577657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls") pod "dns-default-2djq9" (UID: "e089d179-e583-4590-ba4c-39e7d486d707") : secret "dns-default-metrics-tls" not found
Oct 30 12:35:52.647125 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646911    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:35:52.647125 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.646938    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls podName:2be5910f-12fa-46fe-bf9a-ddff510f4d09 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.646932228 +0000 UTC m=+46.686635896 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-bdv44" (UID: "2be5910f-12fa-46fe-bf9a-ddff510f4d09") : secret "samples-operator-tls" not found
Oct 30 12:35:52.747924 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.747891    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:52.748082 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.748037    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:35:52.748082 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.748067    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:52.748183 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.748107    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls podName:b4171e4a-4d6e-4901-983a-600ab76ada45 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:54.748087879 +0000 UTC m=+40.787791555 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lw4gw" (UID: "b4171e4a-4d6e-4901-983a-600ab76ada45") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:35:52.748183 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.748124    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 12:35:52.748183 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.748171    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm podName:b4171e4a-4d6e-4901-983a-600ab76ada45 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:54.748150875 +0000 UTC m=+40.787854550 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lw4gw" (UID: "b4171e4a-4d6e-4901-983a-600ab76ada45") : configmap "kube-rbac-proxy" not found
Oct 30 12:35:52.799765 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.799713    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-tjlfm"]
Oct 30 12:35:52.804041 ip-10-0-142-36 kubenswrapper[2581]: W1030 12:35:52.804011    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaeda6ae_9b69_4cd2_b57b_78dec64c9c7d.slice/crio-aa17ae3fe81ca3f5eb2127624190704b10c0c525c3427f5336204c20921ecb83 WatchSource:0}: Error finding container aa17ae3fe81ca3f5eb2127624190704b10c0c525c3427f5336204c20921ecb83: Status 404 returned error can't find the container with id aa17ae3fe81ca3f5eb2127624190704b10c0c525c3427f5336204c20921ecb83
Oct 30 12:35:52.853621 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.853546    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-29mpd" event={"ID":"a7e45249-4b17-415f-966d-af5321f182d5","Type":"ContainerStarted","Data":"e1429b64292a19b966e55537d5c086910b45ae5b3689a46e95d18b2477b4e7b3"}
Oct 30 12:35:52.854074 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.853776    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:52.855634 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.855545    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tjlfm" event={"ID":"eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d","Type":"ContainerStarted","Data":"aa17ae3fe81ca3f5eb2127624190704b10c0c525c3427f5336204c20921ecb83"}
Oct 30 12:35:52.857385 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.857364    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/1.log"
Oct 30 12:35:52.857752 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.857736    2581 scope.go:122] "RemoveContainer" containerID="962e40dd4a1466cf55a7d8df3804586fbec551f4fb8919a94013f65991c6296b"
Oct 30 12:35:52.857927 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:52.857908    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-8gvm8_openshift-console-operator(b80ffbfe-445f-431c-b8f4-be9fd2c5d094)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8" podUID="b80ffbfe-445f-431c-b8f4-be9fd2c5d094"
Oct 30 12:35:52.859950 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.859922    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-gzkb8" event={"ID":"10a8a53c-d9d6-45aa-936e-48745236bb01","Type":"ContainerStarted","Data":"04ee1ff81bbbba67051c18704d07ac4f336ee35df0bb997e49d8f24716f4b080"}
Oct 30 12:35:52.892258 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.892205    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-29mpd" podStartSLOduration=34.474317048 podStartE2EDuration="37.892193068s" totalImagesPullingTime="3.41787602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:52.8907307 +0000 UTC m=+38.930434391" watchObservedRunningTime="2026-10-30 12:35:52.892193068 +0000 UTC m=+38.931896758"
Oct 30 12:35:52.971291 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:52.971247    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-gzkb8" podStartSLOduration=382.481810933 podStartE2EDuration="6m28.971234596s" totalImagesPullingTime="6.489423663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:29:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:52.970413838 +0000 UTC m=+39.010117526" watchObservedRunningTime="2026-10-30 12:35:52.971234596 +0000 UTC m=+39.010938287"
Oct 30 12:35:54.771587 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:54.771548    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:54.772048 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:54.771721    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:35:54.772048 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:54.771808    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls podName:b4171e4a-4d6e-4901-983a-600ab76ada45 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:58.771787874 +0000 UTC m=+44.811491563 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lw4gw" (UID: "b4171e4a-4d6e-4901-983a-600ab76ada45") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:35:54.772048 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:54.771835    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:54.772048 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:54.771933    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 12:35:54.772048 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:54.771978    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm podName:b4171e4a-4d6e-4901-983a-600ab76ada45 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:58.771966241 +0000 UTC m=+44.811669924 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lw4gw" (UID: "b4171e4a-4d6e-4901-983a-600ab76ada45") : configmap "kube-rbac-proxy" not found
Oct 30 12:35:54.871804 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:54.871771    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tjlfm" event={"ID":"eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d","Type":"ContainerStarted","Data":"50355f048de2a1dd8f3cf412785f26590cd3855663c28cb7f10d21b17ed9e466"}
Oct 30 12:35:54.909015 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:54.908859    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tjlfm" podStartSLOduration=1.4586634410000001 podStartE2EDuration="2.908844107s" totalImagesPullingTime="1.450180666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:35:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:35:54.908296945 +0000 UTC m=+40.948000636" watchObservedRunningTime="2026-10-30 12:35:54.908844107 +0000 UTC m=+40.948547826"
Oct 30 12:35:55.124166 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:55.124138    2581 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:55.124484 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:55.124469    2581 scope.go:122] "RemoveContainer" containerID="962e40dd4a1466cf55a7d8df3804586fbec551f4fb8919a94013f65991c6296b"
Oct 30 12:35:55.124669 ip-10-0-142-36 kubenswrapper[2581]: E1030 12:35:55.124646    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-8gvm8_openshift-console-operator(b80ffbfe-445f-431c-b8f4-be9fd2c5d094)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8" podUID="b80ffbfe-445f-431c-b8f4-be9fd2c5d094"
Oct 30 12:35:55.342323 ip-10-0-142-36 kubenswrapper[2581]: I1030 12:35:55.342209    2581 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 12:35:55.342346 ip-10-0-142-36 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 12:35:55.358030 ip-10-0-142-36 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 12:35:55.358211 ip-10-0-142-36 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 12:35:55.358266 ip-10-0-142-36 systemd[1]: kubelet.service: Consumed 3.029s CPU time.
Oct 30 12:35:55.375266 ip-10-0-142-36 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 12:35:55.452279 ip-10-0-142-36 kubenswrapper[4945]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:35:55.452279 ip-10-0-142-36 kubenswrapper[4945]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 12:35:55.452279 ip-10-0-142-36 kubenswrapper[4945]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:35:55.452279 ip-10-0-142-36 kubenswrapper[4945]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 12:35:55.454608 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454510    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:55.454608 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454589    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:55.454608 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454608    4945 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:55.454608 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454613    4945 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454617    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454623    4945 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454627    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454631    4945 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454635    4945 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454639    4945 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454642    4945 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454646    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454650    4945 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454654    4945 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454658    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454661    4945 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454665    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454668    4945 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454673    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454676    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454680    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454685    4945 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454689    4945 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:55.454877 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454693    4945 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454697    4945 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454701    4945 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454705    4945 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454719    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454724    4945 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454727    4945 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454732    4945 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454736    4945 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454739    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454744    4945 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454748    4945 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454752    4945 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454757    4945 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454761    4945 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454766    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454794    4945 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454800    4945 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454804    4945 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454808    4945 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:55.455775 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454816    4945 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454821    4945 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454827    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454832    4945 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454837    4945 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454841    4945 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454847    4945 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454853    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454857    4945 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454863    4945 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454867    4945 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454871    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454875    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454879    4945 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454884    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454888    4945 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454892    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454896    4945 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454901    4945 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:55.456666 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454905    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454910    4945 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454914    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454918    4945 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454923    4945 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454928    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454932    4945 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454936    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454940    4945 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454945    4945 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454949    4945 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454953    4945 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454958    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454962    4945 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454968    4945 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454972    4945 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454976    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454980    4945 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454985    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454989    4945 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:55.457510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454993    4945 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.454998    4945 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455002    4945 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455006    4945 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455011    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455019    4945 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455026    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455031    4945 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455036    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455040    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455045    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455049    4945 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455054    4945 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455058    4945 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455062    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455066    4945 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455070    4945 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455075    4945 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455079    4945 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:55.458392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455084    4945 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455089    4945 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455094    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455098    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.455102    4945 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456322    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456332    4945 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456337    4945 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456343    4945 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456348    4945 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456352    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456357    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456361    4945 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456366    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456370    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456374    4945 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456379    4945 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456383    4945 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456388    4945 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456393    4945 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:55.459265 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456397    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456401    4945 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456405    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456410    4945 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456414    4945 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456418    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456423    4945 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456427    4945 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456432    4945 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456439    4945 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456445    4945 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456450    4945 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456454    4945 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456458    4945 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456463    4945 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456467    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456471    4945 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456475    4945 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456480    4945 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:55.460166 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456485    4945 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456489    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456495    4945 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456500    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456504    4945 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456509    4945 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456513    4945 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456517    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456522    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456526    4945 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456530    4945 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456535    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456539    4945 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456546    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456550    4945 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456554    4945 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456558    4945 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456562    4945 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456567    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456574    4945 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:55.461042 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456582    4945 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456586    4945 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456591    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456613    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456631    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456636    4945 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456640    4945 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456644    4945 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456649    4945 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456653    4945 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456657    4945 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456661    4945 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456666    4945 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456670    4945 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456674    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456679    4945 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456683    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456687    4945 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456691    4945 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456695    4945 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:55.461567 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456699    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456703    4945 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456707    4945 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456712    4945 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456716    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456725    4945 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456729    4945 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456734    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456738    4945 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456742    4945 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456746    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456750    4945 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456755    4945 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456759    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456764    4945 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456768    4945 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456772    4945 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456776    4945 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456780    4945 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:55.462076 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456785    4945 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456789    4945 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456793    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456797    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456802    4945 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456806    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456810    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456815    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456819    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456824    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456828    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456834    4945 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.456838    4945 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457711    4945 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457722    4945 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457727    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457732    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457736    4945 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457744    4945 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:55.462583 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457748    4945 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457753    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457757    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457761    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457766    4945 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457770    4945 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457774    4945 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457779    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457783    4945 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457788    4945 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457792    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457796    4945 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457801    4945 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457805    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457810    4945 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457814    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457818    4945 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457822    4945 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457826    4945 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457831    4945 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:55.463414 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457835    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457839    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457845    4945 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457850    4945 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457854    4945 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457858    4945 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457863    4945 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457867    4945 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457871    4945 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457875    4945 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457880    4945 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457886    4945 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457891    4945 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457895    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457899    4945 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457903    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457907    4945 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457912    4945 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457916    4945 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457920    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:55.464296 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457924    4945 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457928    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457933    4945 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457937    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457943    4945 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457947    4945 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457951    4945 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457956    4945 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457962    4945 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457968    4945 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457973    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457977    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457981    4945 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457985    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457989    4945 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457995    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.457999    4945 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458003    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458007    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:55.465221 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458011    4945 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458016    4945 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458022    4945 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458028    4945 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458034    4945 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458039    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458043    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458047    4945 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458051    4945 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458056    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458060    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458065    4945 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458069    4945 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458073    4945 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458077    4945 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458081    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458093    4945 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458098    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458102    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458107    4945 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:55.466056 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458134    4945 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458141    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458146    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458151    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458155    4945 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458160    4945 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458164    4945 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458168    4945 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458172    4945 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458179    4945 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458183    4945 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458188    4945 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458193    4945 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458197    4945 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458201    4945 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458206    4945 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458212    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458216    4945 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458221    4945 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458225    4945 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:55.466966 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.458229    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458354    4945 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458371    4945 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458379    4945 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458386    4945 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458392    4945 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458397    4945 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458403    4945 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458409    4945 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458414    4945 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458418    4945 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458424    4945 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458432    4945 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458437    4945 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458442    4945 flags.go:64] FLAG: --cgroup-root=""
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458446    4945 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458452    4945 flags.go:64] FLAG: --client-ca-file=""
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458456    4945 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458461    4945 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458468    4945 flags.go:64] FLAG: --cluster-domain=""
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458473    4945 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458478    4945 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458484    4945 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458490    4945 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 12:35:55.467869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458496    4945 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458501    4945 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458505    4945 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458511    4945 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458516    4945 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458521    4945 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458526    4945 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458530    4945 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458535    4945 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458542    4945 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458546    4945 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458551    4945 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458556    4945 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458560    4945 flags.go:64] FLAG: --enable-server="true"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458565    4945 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458572    4945 flags.go:64] FLAG: --event-burst="100"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458577    4945 flags.go:64] FLAG: --event-qps="50"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458582    4945 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458587    4945 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458592    4945 flags.go:64] FLAG: --eviction-hard=""
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458616    4945 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458622    4945 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458626    4945 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458632    4945 flags.go:64] FLAG: --eviction-soft=""
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458637    4945 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 12:35:55.468932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458642    4945 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458647    4945 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458652    4945 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458657    4945 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458661    4945 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458666    4945 flags.go:64] FLAG: --feature-gates=""
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458673    4945 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458678    4945 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458684    4945 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458689    4945 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458694    4945 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458699    4945 flags.go:64] FLAG: --help="false"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458703    4945 flags.go:64] FLAG: --hostname-override="ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458708    4945 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458713    4945 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458718    4945 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458748    4945 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458754    4945 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458759    4945 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458763    4945 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458768    4945 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458773    4945 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458778    4945 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458783    4945 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 12:35:55.470099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458788    4945 flags.go:64] FLAG: --kube-reserved=""
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458793    4945 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458797    4945 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458805    4945 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458810    4945 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458815    4945 flags.go:64] FLAG: --lock-file=""
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458820    4945 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458825    4945 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458829    4945 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458838    4945 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458842    4945 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458847    4945 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458852    4945 flags.go:64] FLAG: --logging-format="text"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458857    4945 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458862    4945 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458869    4945 flags.go:64] FLAG: --manifest-url=""
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458874    4945 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458881    4945 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458886    4945 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458892    4945 flags.go:64] FLAG: --max-pods="110"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458898    4945 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458903    4945 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458908    4945 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458912    4945 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458917    4945 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 12:35:55.471127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458922    4945 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458926    4945 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458937    4945 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458942    4945 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458947    4945 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458952    4945 flags.go:64] FLAG: --pod-cidr=""
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458957    4945 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458962    4945 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458966    4945 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458971    4945 flags.go:64] FLAG: --port="10250"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458978    4945 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458983    4945 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00a767a3fd94df4b3"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458988    4945 flags.go:64] FLAG: --qos-reserved=""
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458994    4945 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.458998    4945 flags.go:64] FLAG: --register-node="true"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459003    4945 flags.go:64] FLAG: --register-with-taints=""
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459009    4945 flags.go:64] FLAG: --registry-burst="10"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459016    4945 flags.go:64] FLAG: --registry-qps="5"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459021    4945 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459026    4945 flags.go:64] FLAG: --reserved-memory=""
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459032    4945 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459037    4945 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459042    4945 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459048    4945 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459052    4945 flags.go:64] FLAG: --runonce="false"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459057    4945 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 12:35:55.472363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459062    4945 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459067    4945 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459072    4945 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459077    4945 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459082    4945 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459086    4945 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459091    4945 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459096    4945 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459100    4945 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459106    4945 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459111    4945 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459116    4945 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459121    4945 flags.go:64] FLAG: --system-cgroups=""
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459125    4945 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459133    4945 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459138    4945 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459144    4945 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459151    4945 flags.go:64] FLAG: --tls-min-version=""
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459155    4945 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459160    4945 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459164    4945 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459169    4945 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459174    4945 flags.go:64] FLAG: --v="2"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459183    4945 flags.go:64] FLAG: --version="false"
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459189    4945 flags.go:64] FLAG: --vmodule=""
Oct 30 12:35:55.473199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459195    4945 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459201    4945 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459406    4945 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459413    4945 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459417    4945 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459423    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459427    4945 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459431    4945 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459436    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459440    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459444    4945 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459452    4945 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459459    4945 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459464    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459469    4945 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459474    4945 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459478    4945 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459482    4945 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459487    4945 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459491    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:55.473838 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459495    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459500    4945 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459504    4945 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459510    4945 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459515    4945 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459520    4945 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459524    4945 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459528    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459532    4945 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459537    4945 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459543    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459548    4945 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459553    4945 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459557    4945 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459561    4945 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459566    4945 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459570    4945 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459574    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459579    4945 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459583    4945 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:55.474360 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459588    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459592    4945 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459613    4945 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459618    4945 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459622    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459627    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459631    4945 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459635    4945 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459639    4945 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459643    4945 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459648    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459651    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459656    4945 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459660    4945 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459664    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459671    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459676    4945 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459680    4945 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459684    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:55.474869 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459688    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459692    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459697    4945 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459702    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459707    4945 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459711    4945 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459715    4945 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459720    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459725    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459729    4945 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459733    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459738    4945 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459743    4945 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459747    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459751    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459756    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459760    4945 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459764    4945 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459769    4945 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459773    4945 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:55.475362 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459777    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459781    4945 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459785    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459789    4945 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459793    4945 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459798    4945 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459805    4945 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459811    4945 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459819    4945 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459824    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459829    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459833    4945 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459839    4945 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459843    4945 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459847    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459853    4945 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459857    4945 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459862    4945 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459866    4945 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:55.475943 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459870    4945 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459874    4945 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459879    4945 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459883    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459887    4945 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459891    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459896    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459901    4945 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459905    4945 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.459909    4945 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.459950    4945 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460004    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460011    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460016    4945 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460021    4945 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460025    4945 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:55.476433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460029    4945 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460034    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460038    4945 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460043    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460048    4945 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460055    4945 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460060    4945 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460066    4945 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460073    4945 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460078    4945 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460082    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460086    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460091    4945 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460097    4945 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460102    4945 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460106    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460110    4945 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460114    4945 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460118    4945 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:55.476901 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460123    4945 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460127    4945 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460131    4945 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460136    4945 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460140    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460145    4945 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460149    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460153    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460157    4945 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460161    4945 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460166    4945 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460170    4945 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460174    4945 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460178    4945 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460181    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460185    4945 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460189    4945 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460194    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460198    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460202    4945 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:55.477468 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460209    4945 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460215    4945 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460220    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460225    4945 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460230    4945 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460234    4945 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460240    4945 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460245    4945 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460249    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460254    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460258    4945 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460262    4945 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460267    4945 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460271    4945 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460275    4945 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460279    4945 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460283    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460290    4945 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460294    4945 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460299    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:55.478038 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460304    4945 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460309    4945 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460313    4945 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460317    4945 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460322    4945 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460326    4945 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460330    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460335    4945 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460339    4945 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460343    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460347    4945 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460352    4945 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460356    4945 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460361    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460365    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460369    4945 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460373    4945 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460378    4945 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460384    4945 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460388    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:55.478690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460392    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460396    4945 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460401    4945 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460405    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460409    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460414    4945 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460419    4945 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460424    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460428    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460433    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460437    4945 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460442    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460446    4945 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460450    4945 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460454    4945 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460459    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460465    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460469    4945 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460473    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:55.479158 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460478    4945 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460482    4945 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.460486    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.465697    4945 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.465714    4945 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.465737    4945 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.465744    4945 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465837    4945 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465845    4945 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465850    4945 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465855    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465861    4945 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465865    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465870    4945 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465874    4945 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465878    4945 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465882    4945 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465887    4945 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465892    4945 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:55.479627 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465896    4945 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465900    4945 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465904    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465909    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465921    4945 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465936    4945 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465942    4945 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465947    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465951    4945 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465956    4945 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465960    4945 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465964    4945 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465968    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465972    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465977    4945 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465982    4945 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465986    4945 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465990    4945 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465994    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:55.480094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.465998    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466003    4945 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466007    4945 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466011    4945 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466015    4945 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466020    4945 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466025    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466029    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466033    4945 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466037    4945 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466041    4945 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466045    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466049    4945 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466053    4945 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466058    4945 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466062    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466066    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466070    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466083    4945 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:55.480547 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466099    4945 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466106    4945 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466111    4945 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466116    4945 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466121    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466125    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466130    4945 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466134    4945 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466138    4945 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466142    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466146    4945 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466151    4945 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466155    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466159    4945 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466163    4945 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466167    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466171    4945 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466175    4945 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466180    4945 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466184    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466189    4945 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:55.481094 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466193    4945 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466197    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466201    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466206    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466210    4945 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466214    4945 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466219    4945 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466223    4945 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466227    4945 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466231    4945 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466235    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466248    4945 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466252    4945 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466257    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466261    4945 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466265    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466269    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466273    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466278    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:55.481773 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466282    4945 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466286    4945 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466291    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466295    4945 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466299    4945 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466303    4945 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466307    4945 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466311    4945 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466315    4945 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466320    4945 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466324    4945 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466329    4945 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466333    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466337    4945 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466341    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466345    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:55.482417 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.466387    4945 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466707    4945 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466716    4945 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466722    4945 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466726    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466730    4945 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466734    4945 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466738    4945 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466742    4945 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466756    4945 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466762    4945 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466766    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466770    4945 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466774    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466778    4945 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466782    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466786    4945 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466790    4945 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466795    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:55.482957 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466799    4945 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466803    4945 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466808    4945 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466812    4945 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466816    4945 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466823    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466827    4945 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466834    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466839    4945 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466846    4945 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466854    4945 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466860    4945 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466864    4945 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466870    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466874    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466879    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466883    4945 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466888    4945 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466892    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:55.483392 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466897    4945 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466901    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466905    4945 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466909    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466924    4945 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466929    4945 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466941    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466946    4945 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466950    4945 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466954    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466958    4945 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466962    4945 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466967    4945 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466970    4945 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466975    4945 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466979    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466982    4945 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466987    4945 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466993    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:55.483872 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.466997    4945 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467004    4945 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467008    4945 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467012    4945 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467017    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467021    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467025    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467030    4945 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467034    4945 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467038    4945 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467043    4945 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467047    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467051    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467056    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467060    4945 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467064    4945 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467068    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467073    4945 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467085    4945 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467090    4945 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:55.484312 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467095    4945 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467099    4945 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467103    4945 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467107    4945 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467112    4945 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467116    4945 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467120    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467124    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467128    4945 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467133    4945 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467137    4945 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467147    4945 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467152    4945 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467159    4945 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467163    4945 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467168    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467172    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467177    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467181    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467185    4945 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:55.484953 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467189    4945 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467194    4945 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467198    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467202    4945 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467206    4945 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467211    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467215    4945 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467219    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467223    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467228    4945 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.467267    4945 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467350    4945 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467357    4945 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467361    4945 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467365    4945 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467370    4945 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 12:35:55.485434 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467374    4945 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467379    4945 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467383    4945 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467387    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467392    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467396    4945 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467400    4945 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467405    4945 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467412    4945 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467421    4945 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467426    4945 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467430    4945 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467435    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467439    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467443    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467448    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467452    4945 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467456    4945 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467461    4945 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467465    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 12:35:55.485843 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467470    4945 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467474    4945 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467478    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467482    4945 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467486    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467491    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467495    4945 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467499    4945 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467514    4945 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467519    4945 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467523    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467527    4945 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467531    4945 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467536    4945 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467540    4945 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467544    4945 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467548    4945 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467552    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467557    4945 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 12:35:55.486358 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467561    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467567    4945 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467573    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467577    4945 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467581    4945 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467585    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467589    4945 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467609    4945 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467614    4945 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467618    4945 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467623    4945 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467627    4945 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467631    4945 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467635    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467641    4945 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467648    4945 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467654    4945 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467660    4945 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467666    4945 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 12:35:55.486832 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467670    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467675    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467679    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467692    4945 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467696    4945 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467701    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467705    4945 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467709    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467713    4945 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467718    4945 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467722    4945 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467726    4945 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467730    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467735    4945 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467741    4945 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467746    4945 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467751    4945 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467755    4945 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467759    4945 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 12:35:55.487280 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467764    4945 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467768    4945 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467772    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467776    4945 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467781    4945 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467785    4945 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467789    4945 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467795    4945 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467799    4945 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467803    4945 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467807    4945 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467812    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467816    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467820    4945 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467824    4945 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467828    4945 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467832    4945 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467845    4945 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467849    4945 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467855    4945 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 12:35:55.487864 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467859    4945 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 12:35:55.488728 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467864    4945 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 12:35:55.488728 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467868    4945 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 12:35:55.488728 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:35:55.467872    4945 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 12:35:55.488728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.467942    4945 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 12:35:55.488728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.469445    4945 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 12:35:55.488728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.469513    4945 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 12:35:55.488728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.469825    4945 server.go:1014] "Starting client certificate rotation"
Oct 30 12:35:55.488728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.470001    4945 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 12:35:55.488728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.470052    4945 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 12:30:15 +0000 UTC" deadline="2028-08-12 19:12:27.793653274 +0000 UTC"
Oct 30 12:35:55.488728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.470125    4945 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15654h36m32.323532817s"
Oct 30 12:35:55.488728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.470709    4945 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 12:35:55.488728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.473947    4945 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 12:35:55.488728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.475450    4945 log.go:25] "Validated CRI v1 runtime API"
Oct 30 12:35:55.488728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.477093    4945 log.go:25] "Validated CRI v1 image API"
Oct 30 12:35:55.488728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.477925    4945 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 12:35:55.489293 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.479833    4945 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 f05e0c01-eaf2-4645-89d4-33cbcd4b548c:/dev/nvme0n1p4 fff9e7d8-1832-4521-9d95-6334c72f9fc4:/dev/nvme0n1p3]
Oct 30 12:35:55.489396 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.479853    4945 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/09fceefd9630cf2a164a77f3bb0545dadde776dfacf0dc5e986919d2ee3116c5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/09fceefd9630cf2a164a77f3bb0545dadde776dfacf0dc5e986919d2ee3116c5/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1af18b9b9ea3cf77550be3bc00adbc0fb260e197d40cdf8e54ab968dc6323fff/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1af18b9b9ea3cf77550be3bc00adbc0fb260e197d40cdf8e54ab968dc6323fff/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3351f698fa606515af8dcb112bf4cea5cd1434a7232fdb021c7abf39525e78b1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3351f698fa606515af8dcb112bf4cea5cd1434a7232fdb021c7abf39525e78b1/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3b7b2649a9fbcd99e7bf75a6aac80cf8db43aec6c649374e9a5603007884136f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3b7b2649a9fbcd99e7bf75a6aac80cf8db43aec6c649374e9a5603007884136f/userdata/shm major:0 minor:228 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3cd6393cdf0c12b4dadc6f89839a211107470255881edd2e736a1dfef317e7fd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3cd6393cdf0c12b4dadc6f89839a211107470255881edd2e736a1dfef317e7fd/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4abb3ff147c2ca169b46c138813140b41dbe52d3eab89d6b4a2c31371d0197d7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4abb3ff147c2ca169b46c138813140b41dbe52d3eab89d6b4a2c31371d0197d7/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4d6911b025d9e88987f3642ba349b3c4e1559445b1f9f5408d99e1a9ed1e51a9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4d6911b025d9e88987f3642ba349b3c4e1559445b1f9f5408d99e1a9ed1e51a9/userdata/shm major:0 minor:243 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/57296fb69fdd6aa9c6628c3621c1b66d19e6d3956c607a31461b6910e5294233/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/57296fb69fdd6aa9c6628c3621c1b66d19e6d3956c607a31461b6910e5294233/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/621ace500d4f75ef9c887619ca34581f60018c1b26a94ae2a4dcaf17ceb7fa74/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/621ace500d4f75ef9c887619ca34581f60018c1b26a94ae2a4dcaf17ceb7fa74/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/62c6703912aecec773d95240640baeb4395beeab94b0ac8e1d379492343ac48c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/62c6703912aecec773d95240640baeb4395beeab94b0ac8e1d379492343ac48c/userdata/shm major:0 minor:81 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/69652a499a9fef069224a36d8831b8831c15ee6dde86a929b4e48599cf0044f2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/69652a499a9fef069224a36d8831b8831c15ee6dde86a929b4e48599cf0044f2/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/757c56165d7a422d29db3395a4755e5bc5c7251c3dadac21fa6fb98b3fe8cca9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/757c56165d7a422d29db3395a4755e5bc5c7251c3dadac21fa6fb98b3fe8cca9/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8a9cb48c43d1cf1ab491ef872fe88365a0da7a311fa8f4bb318400164ef97043/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8a9cb48c43d1cf1ab491ef872fe88365a0da7a311fa8f4bb318400164ef97043/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/918f40e9b236196597ce1d674a12bd818bd975806363262c6f485da82c1a5346/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/918f40e9b236196597ce1d674a12bd818bd975806363262c6f485da82c1a5346/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/966dcd2556e1ec1346a605685f267af7369016b91467bf9b088d1fd25d4ad694/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/966dcd2556e1ec1346a605685f267af7369016b91467bf9b088d1fd25d4ad694/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9ed57772b7f777bf45d80ea1035ae4a8c4e3fc0876275655ceda617b1849ba7f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9ed57772b7f777bf45d80ea1035ae4a8c4e3fc0876275655ceda617b1849ba7f/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/aa17ae3fe81ca3f5eb2127624190704b10c0c525c3427f5336204c20921ecb83/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/aa17ae3fe81ca3f5eb2127624190704b10c0c525c3427f5336204c20921ecb83/userdata/shm major:0 minor:336 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ab1eba861addee696d5063000eae19b0fb3f0aa5c112d7ab33a2732fc5767e8e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ab1eba861addee696d5063000eae19b0fb3f0aa5c112d7ab33a2732fc5767e8e/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c2ce37edc542d23f0715646c0c806e458b1cf6afdb37838f35b00e122f08fb4a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c2ce37edc542d23f0715646c0c806e458b1cf6afdb37838f35b00e122f08fb4a/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c88512809f317fb4be946430d40291f8d55c07f7231005bd8d88bd9496953990/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c88512809f317fb4be946430d40291f8d55c07f7231005bd8d88bd9496953990/userdata/shm major:0 minor:294 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dc2f830ae39d642b0c9b00c36a1f943093a33bb54659bc00bb35c1fd34a304e2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dc2f830ae39d642b0c9b00c36a1f943093a33bb54659bc00bb35c1fd34a304e2/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e73ede3436e8237d52c9d329de9e46c44e189332e305ccdad50cf1eb4df1dbdd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e73ede3436e8237d52c9d329de9e46c44e189332e305ccdad50cf1eb4df1dbdd/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fb8a16464c99266ad50f6c092a3fcb92ed6c2701b65333a20f9d5a63b1271d33/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fb8a16464c99266ad50f6c092a3fcb92ed6c2701b65333a20f9d5a63b1271d33/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/06910518-58db-450f-a5ee-521ac8f51e1a/volumes/kubernetes.io~projected/kube-api-access-slh7l:{mountpoint:/var/lib/kubelet/pods/06910518-58db-450f-a5ee-521ac8f51e1a/volumes/kubernetes.io~projected/kube-api-access-slh7l major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/10337f54-95e9-42cd-a73f-16817fc1cb96/volumes/kubernetes.io~projected/kube-api-access-p5nr5:{mountpoint:/var/lib/kubelet/pods/10337f54-95e9-42cd-a73f-16817fc1cb96/volumes/kubernetes.io~projected/kube-api-access-p5nr5 major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/10337f54-95e9-42cd-a73f-16817fc1cb96/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/10337f54-95e9-42cd-a73f-16817fc1cb96/volumes/kubernetes.io~secret/cert major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/10a8a53c-d9d6-45aa-936e-48745236bb01/volumes/kubernetes.io~projected/kube-api-access-jknfr:{mountpoint:/var/lib/kubelet/pods/10a8a53c-d9d6-45aa-936e-48745236bb01/volumes/kubernetes.io~projected/kube-api-access-jknfr major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1263e83a-e0fd-4eb4-8141-5026aec6a631/volumes/kubernetes.io~projected/kube-api-access-825sw:{mountpoint:/var/lib/kubelet/pods/1263e83a-e0fd-4eb4-8141-5026aec6a631/volumes/kubernetes.io~projected/kube-api-access-825sw major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/15bb3274-f845-4851-90ae-f4ab08e685c8/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/15bb3274-f845-4851-90ae-f4ab08e685c8/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/18fbde11-0050-4b55-bc23-a4e5e76ddc81/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/18fbde11-0050-4b55-bc23-a4e5e76ddc81/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/18fbde11-0050-4b55-bc23-a4e5e76ddc81/volumes/kubernetes.io~projected/kube-api-access-ppcp2:{mountpoint:/var/lib/kubelet/pods/18fbde11-0050-4b55-bc23-a4e5e76ddc81/volumes/kubernetes.io~projected/kube-api-access-ppcp2 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/18fbde11-0050-4b55-bc23-a4e5e76ddc81/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/18fbde11-0050-4b55-bc23-a4e5e76ddc81/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2758dbad-1f8a-4503-9dc2-6181518daeaf/volumes/kubernetes.io~projected/kube-api-access-q8xqw:{mountpoint:/var/lib/kubelet/pods/2758dbad-1f8a-4503-9dc2-6181518daeaf/volumes/kubernetes.io~projected/kube-api-access-q8xqw major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2be5910f-12fa-46fe-bf9a-ddff510f4d09/volumes/kubernetes.io~projected/kube-api-access-2khhs:{mountpoint:/var/lib/kubelet/pods/2be5910f-12fa-46fe-bf9a-ddff510f4d09/volumes/kubernetes.io~projected/kube-api-access-2khhs major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/336de054-d674-4f26-81d8-980b11e27ace/volumes/kubernetes.io~projected/kube-api-access-5fbjv:{mountpoint:/var/lib/kubelet/pods/336de054-d674-4f26-81d8-980b11e27ace/volumes/kubernetes.io~projected/kube-api-access-5fbjv major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3a2f595c-6efe-4b69-81bf-074985a6ddde/volumes/kubernetes.io~projected/kube-api-access-fn8lm:{mountpoint:/var/lib/kubelet/pods/3a2f595c-6efe-4b69-81bf-074985a6ddde/volumes/kubernetes.io~projected/kube-api-access-fn8lm major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3ab05e61-6a22-4ced-abec-a528a15f9ba0/volumes/kubernetes.io~projected/kube-api-access-f86qq:{mountpoint:/var/lib/kubelet/pods/3ab05e61-6a22-4ced-abec-a528a15f9ba0/volumes/kubernetes.io~projected/kube-api-access-f86qq major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4b56b071-d626-4d8d-bba6-f5f69fa95db0/volumes/kubernetes.io~projected/kube-api-access-vhm96:{mountpoint:/var/lib/kubelet/pods/4b56b071-d626-4d8d-bba6-f5f69fa95db0/volumes/kubernetes.io~projected/kube-api-access-vhm96 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4f13aca3-562e-406b-a154-8755b1bf7d15/volumes/kubernetes.io~projected/kube-api-access-cfh24:{mountpoint:/var/lib/kubelet/pods/4f13aca3-562e-406b-a154-8755b1bf7d15/volumes/kubernetes.io~projected/kube-api-access-cfh24 major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4f13aca3-562e-406b-a154-8755b1bf7d15/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/4f13aca3-562e-406b-a154-8755b1bf7d15/volumes/kubernetes.io~secret/serving-cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/60e62efb-7c65-41f3-87d4-0805eec8f7e9/volumes/kubernetes.io~projected/kube-api-access-xk2ml:{mountpoint:/var/lib/kubelet/pods/60e62efb-7c65-41f3-87d4-0805eec8f7e9/volumes/kubernetes.io~projected/kube-api-access-xk2ml major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/60e62efb-7c65-41f3-87d4-0805eec8f7e9/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/60e62efb-7c65-41f3-87d4-0805eec8f7e9/volumes/kubernetes.io~secret/serving-cert major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/79f0f110-a04f-4b35-8eba-c8f0469b62a1/volumes/kubernetes.io~projected/kube-api-access-bzdfs:{mountpoint:/var/lib/kubelet/pods/79f0f110-a04f-4b35-8eba-c8f0469b62a1/volumes/kubernetes.io~projected/kube-api-access-bzdfs major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce/volumes/kubernetes.io~secret/agent-certs major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a594de5b-9748-4355-86b8-95ca7dc9dda1/volumes/kubernetes.io~projected/kube-api-access-bkj5v:{mountpoint:/var/lib/kubelet/pods/a594de5b-9748-4355-86b8-95ca7dc9dda1/volumes/kubernetes.io~projected/kube-api-access-bkj5v major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a7e45249-4b17-415f-966d-af5321f182d5/volumes/kubernetes.io~projected/kube-api-access-lshc4:{mountpoint:/var/lib/kubelet/pods/a7e45249-4b17-415f-966d-af5321f182d5/volumes/kubernetes.io~projected/kube-api-access-lshc4 major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aba04aa0-2bee-47e2-9f36-8790309d6e06/volumes/kubernetes.io~projected/kube-api-access-xftjm:{mountpoint:/var/lib/kubelet/pods/aba04aa0-2bee-47e2-9f36-8790309d6e06/volumes/kubernetes.io~projected/kube-api-access-xftjm major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aba04aa0-2bee-47e2-9f36-8790309d6e06/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/aba04aa0-2bee-47e2-9f36-8790309d6e06/volumes/kubernetes.io~secret/serving-cert major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b4171e4a-4d6e-4901-983a-600ab76ada45/volumes/kubernetes.io~projected/kube-api-access-4ddnv:{mountpoint:/var/lib/kubelet/pods/b4171e4a-4d6e-4901-983a-600ab76ada45/volumes/kubernetes.io~projected/kube-api-access-4ddnv major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b80ffbfe-445f-431c-b8f4-be9fd2c5d094/volumes/kubernetes.io~projected/kube-api-access-9b6xs:{mountpoint:/var/lib/kubelet/pods/b80ffbfe-445f-431c-b8f4-be9fd2c5d094/volumes/kubernetes.io~projected/kube-api-access-9b6xs major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b80ffbfe-445f-431c-b8f4-be9fd2c5d094/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/b80ffbfe-445f-431c-b8f4-be9fd2c5d094/volumes/kubernetes.io~secret/serving-cert major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bbb14793-e4b1-4912-9ce2-008cba7ce1d4/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/bbb14793-e4b1-4912-9ce2-008cba7ce1d4/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bbb14793-e4b1-4912-9ce2-008cba7ce1d4/volumes/kubernetes.io~projected/kube-api-access-k2v9l:{mountpoint:/var/lib/kubelet/pods/bbb14793-e4b1-4912-9ce2-008cba7ce1d4/volumes/kubernetes.io~projected/kube-api-access-k2v9l major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bbb14793-e4b1-4912-9ce2-008cba7ce1d4/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/bbb14793-e4b1-4912-9ce2-008cba7ce1d4/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bbb14793-e4b1-4912-9ce2-008cba7ce1d4/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/bbb14793-e4b1-4912-9ce2-008cba7ce1d4/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cc28fe61-b116-4a8b-ad04-123d62365a02/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/cc28fe61-b116-4a8b-ad04-123d62365a02/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cc28fe61-b116-4a8b-ad04-123d62365a02/volumes/kubernetes.io~projected/kube-api-access-2d7qp:{mountpoint:/var/lib/kubelet/pods/cc28fe61-b116-4a8b-ad04-123d62365a02/volumes/kubernetes.io~projected/kube-api-access-2d7qp major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cc28fe61-b116-4a8b-ad04-123d62365a02/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/cc28fe61-b116-4a8b-ad04-123d62365a02/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cc28fe61-b116-4a8b-ad04-123d62365a02/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/cc28fe61-b116-4a8b-ad04-123d62365a02/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d31facfa-368e-4417-b9ce-30c3a495f650/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/d31facfa-368e-4417-b9ce-30c3a495f650/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d31facfa-368e-4417-b9ce-30c3a495f650/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/d31facfa-368e-4417-b9ce-30c3a495f650/volumes/kubernetes.io~empty-dir/tmp major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d31facfa-368e-4417-b9ce-30c3a495f650/volumes/kubernetes.io~projected/kube-api-access-8bm4p:{mountpoint:/var/lib/kubelet/pods/d31facfa-368e-4417-b9ce-30c3a495f650/volumes/kubernetes.io~projected/kube-api-access-8bm4p major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d81730d3-53ed-40b8-a512-348fc042f821/volumes/kubernetes.io~projected/kube-api-access-sfwdk:{mountpoint:/var/lib/kubelet/pods/d81730d3-53ed-40b8-a512-348fc042f821/volumes/kubernetes.io~projected/kube-api-access-sfwdk major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d81730d3-53ed-40b8-a512-348fc042f821/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/d81730d3-53ed-40b8-a512-348fc042f821/volumes/kubernetes.io~secret/default-certificate major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d81730d3-53ed-40b8-a512-348fc042f821/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/d81730d3-53ed-40b8-a512-348fc042f821/volumes/kubernetes.io~secret/stats-auth major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0/volumes/kubernetes.io~projected/kube-api-access-q6wnt:{mountpoint:/var/lib/kubelet/pods/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0/volumes/kubernetes.io~projected/kube-api-access-q6wnt major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e089d179-e583-4590-ba4c-39e7d486d707/volumes/kubernetes.io~projected/kube-api-access-rk26b:{mountpoint:/var/lib/kubelet/pods/e089d179-e583-4590-ba4c-39e7d486d707/volumes/kubernetes.io~projected/kube-api-access-rk26b major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e30d5797-e034-4806-be4f-54f54698920c/volumes/kubernetes.io~projected/kube-api-access-52p8s:{mountpoint:/var/lib/kubelet/pods/e30d5797-e034-4806-be4f-54f54698920c/volumes/kubernetes.io~projected/kube-api-access-52p8s major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e937e696-f801-42e7-9622-d774cca50028/volumes/kubernetes.io~projected/kube-api-access-pmn28:{mountpoint:/var/lib/kubelet/pods/e937e696-f801-42e7-9622-d774cca50028/volumes/kubernetes.io~projected/kube-api-access-pmn28 major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d/volumes/kubernetes.io~projected/kube-api-access-h54zx:{mountpoint:/var/lib/kubelet/pods/eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d/volumes/kubernetes.io~projected/kube-api-access-h54zx major:0 minor:329 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/9a7f1232f6d323655fc5216b0ba41d0068155367f4372be4b77ed5301df6f77c/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/919cbc5fa3666148ddfb8586d8bffa38e4b86b9d0982a3f898d5b31c521ac2cc/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/b78f392394321ee2b090e00570813e95e4aeab9e2140a2d3e64d92d1fd7ca5a6/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/198c2cf84358178144ff94eb3a37dc178a24821629fda187219614a784ec8358/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/073c5925482b77743ccc26ad997f883387d96ddd3f944f3c4d282db84b47e48e/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/d7c8842b6d7735cf3a492b88bfc1d3ac45ebcc2f47900f4dbc0425b7c8d4794b/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/6289b3803613e67e97542930d3e222bfce0f5b512ed8c5dc913bfaa086278af5/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/2478631c62022f1e63ffb5558d3c4da5b576011a78dfc6eec4062642983b2df7/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/54a0da8f9979749299431e891755447edc86f5b5e113b30edc7a9e4104876163/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/d4c37f15eb94869de0c1244d169157d657e07bce6f377832da399b11f89730ac/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-136:{mountpoint:/var/lib/containers/storage/overlay/18d61714d1dbcdd49d4f40dd2e493c937451b9e684bc7cd13751c55f014ded44/merged major:0 minor:136 fsType:overlay blockSize:0} overlay_0-143:{mountpoint:/var/lib/containers/storage/overlay/a3fd261f959d41176535cb90c373d43b38df8574f82543414ed0418e622d9b20/merged major:0 minor:143 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/dc3e8e991d5c5e025d8643a07d5943239f0d242480a23ed478ecf4a7efb4a860/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/460aa99366ef1b0203c89238bcac3baed4940b76b58025954ba55d8d8c914b9d/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/7098ca41090d1394d595b5649903efe08ff40cd83278b112c9ee101dec345faa/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/cf5d0a83884c833b9c3dd9003b622531208d0240323a422b6cf561fc10e5a6dc/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/981a200a0b6923c2614f5dc6fc01d16029bb3a02330c3906ae27fc27dad51f5e/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/d1eec5c48c905b567ed804572d80a434c6c16d39f480dfdad86a003be9b2decc/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/199565a4734ca6f05aefb9a75b25079ed3616f703e74663c62711eedd18ee2be/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/6e750fb6e38e3e9f7ff1f57e5988e2f1d356b40d06d39962ae48fb4a306831bf/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/b1f248c65468f45224c82478ffd3fe577a796ba61b17b3817dc1123adfbccc72/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/b55514dc7df8e7bf00c29318aaba5d0d835b7f0ebe471a40a71e131f2c932851/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/5f2e318c68cffe040e978d893e5649097fb404a737430f892608997ec0ee3b39/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/c20eea3410b2b217e679afe02e82e65956eb0dfaa0a5baded581b3f9322f37a1/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/b193d6e5a35ed41ee66d1c3796d2a73e1cdaed9ad3f5e46753f2cde561718a2d/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/360bf83d8f85e1e47ce4ff374a907a4108e72507b38ae42c7f5948111a15d0a9/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/289b36dbb1c6bff18b9bf5cd78c6abc0ded74547cd7cbc71749bc534a7771fcd/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/e51fead67d6745f6f236fe0d70bf268777f9e0d9aa26017393d5f0655dcfa903/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/8040bf347f363bf519f60bac4311f629fbb3c102aa3bee4adc31f8376f06fd66/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/0c345558f90474608b2ad133910c1c7a0d3b7d9f0d8e0818d32380dfa2be1c60/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-269:{mountpoint:/var/lib/containers/storage/overlay/d8f35b2d5793604f89891a8822dcdca13149936872ea6e50649d330cf4c88245/merged major:0 minor:269 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/180912e026510355a3740b32c1f36cb0c320c07f7028e3f27b29cdc6a6e94d5e/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/7337340ba30afbc1b5144e88ba28417dd981b738aee08916d68eb8879263f446/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/faabf09eb1dcb93f612ba96d70783a1865c10c105146ef2b85a2d0a2a922a7e5/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/6b8f4d7d01fe8b7f7511811adab4e1ac013e688372768f2756e661205ab35090/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/93b47c3ce2b687782fe5cd36246636af82ce17636375ee1a1990a1dcea551668/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/c92e75d4a0ef25c73138b8d22fa4a2fe365a3c8874db739bcee68dc5fb821e58/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/a1a0d082e1b32fcea622ab9e29cd5537bcf1d546cd55a4194aee61e40d505e98/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/0616556066d7c0c9c4ddcd596c5b8f9aa05eb8bb51b95362c48a4fde6a1dadd8/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/48be6ca261b520519614fbd37aa86b5b798c603ac742269319c07de5ec979962/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/06c60fc8d6d2ec29e83f7d27783f335b847ce5727ba25ad8a42f930573f31eef/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-322:{mountpoint:/var/lib/containers/storage/overlay/bdb59d4910a23cc967a24c4d451590cbb726c4760b8cfe1b085dd03859f2e531/merged major:0 minor:322 fsType:overlay blockSize:0} overlay_0-334:{mountpoint:/var/lib/containers/storage/overlay/f0f468ac0e11f19adde8a8322dfcc31fc66ea84c9aa7cd4c0d998f4e532606b0/merged major:0 minor:334 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/1df233d2c6fddce2c2c143c7b7e2c107b788add9a300e6a6d7e9045dbfb621a2/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/4782c13c689a6758dcc3d4c577657a5af583f3e225f6f9c6b9b1ec06ab439139/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/62db0b4bef550dfd2445a64b7482c941f6501aa76fa765a46e5039f308dbeeb2/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/058b0d48bb3712421f6b9eeae639b481e83acc3f7d4d8d1f753ccd449609b997/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/a1ec349e20834db259430b6b2b587bbb197ed33c2308fcc22e44d3fa26ca08a4/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-71:{mountpoint:/var/lib/containers/storage/overlay/8ac7590b28a82334d0bb53030a78a57494e14e15c091b07e2015fa42b76bd005/merged major:0 minor:71 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/675367fa93b46de38ba7c54a45f4d3569c29904bbca6b7355002d91e4ae2d607/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/4b618dfcd4e24446e1d2027afcf9a92aaf5fa2f9c6c3d87d649a1791110fe967/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/8f7b7a1cc0a1f60af7ecfe46a5cf7f6080ca2e910f00c287466c51bd8ae19f5d/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/30dba072188be3fe2522b1425c092d503fe7af70e009b2490a6723c4ebd5e71f/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/ee2d51bd89f72a95c795ea7822c78ffd753be3f506d749d84b372e39281df7d8/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/d6ea494014e92ac11d70a340d63383f54ded6028098850be99a4836b75456199/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/3276bf58a153777d00c75dae1056ba6ea3c6c4cf172832f1158d37a8eb736d5b/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.487243    4945 manager.go:217] Machine: {Timestamp:2026-10-30 12:35:55.486797614 +0000 UTC m=+0.064746694 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3111702 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25e17718a8a609f0a32a2d110d1554 SystemUUID:ec25e177-18a8-a609-f0a3-2a2d110d1554 BootID:dfcad087-3a8d-4dd6-b5fe-b0735943db67 Filesystems:[{Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c2ce37edc542d23f0715646c0c806e458b1cf6afdb37838f35b00e122f08fb4a/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4f13aca3-562e-406b-a154-8755b1bf7d15/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cc28fe61-b116-4a8b-ad04-123d62365a02/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:209 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4b56b071-d626-4d8d-bba6-f5f69fa95db0/volumes/kubernetes.io~projected/kube-api-access-vhm96 DeviceMajor:0 DeviceMinor:65 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/57296fb69fdd6aa9c6628c3621c1b66d19e6d3956c607a31461b6910e5294233/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e73ede3436e8237d52c9d329de9e46c44e189332e305ccdad50cf1eb4df1dbdd/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3351f698fa606515af8dcb112bf4cea5cd1434a7232fdb021c7abf39525e78b1/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/15bb3274-f845-4851-90ae-f4ab08e685c8/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:288 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e30d5797-e034-4806-be4f-54f54698920c/volumes/kubernetes.io~projected/kube-api-access-52p8s DeviceMajor:0 DeviceMinor:212 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-322 DeviceMajor:0 DeviceMinor:322 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/cc28fe61-b116-4a8b-ad04-123d62365a02/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:208 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/336de054-d674-4f26-81d8-980b11e27ace/volumes/kubernetes.io~projected/kube-api-access-5fbjv DeviceMajor:0 DeviceMinor:218 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3b7b2649a9fbcd99e7bf75a6aac80cf8db43aec6c649374e9a5603007884136f/userdata/shm DeviceMajor:0 DeviceMinor:228 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/10337f54-95e9-42cd-a73f-16817fc1cb96/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:206 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bbb14793-e4b1-4912-9ce2-008cba7ce1d4/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:214 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/966dcd2556e1ec1346a605685f267af7369016b91467bf9b088d1fd25d4ad694/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-269 DeviceMajor:0 DeviceMinor:269 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2758dbad-1f8a-4503-9dc2-6181518daeaf/volumes/kubernetes.io~projected/kube-api-access-q8xqw DeviceMajor:0 DeviceMinor:64 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-71 DeviceMajor:0 DeviceMinor:71 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e937e696-f801-42e7-9622-d774cca50028/volumes/kubernetes.io~projected/kube-api-access-pmn28 DeviceMajor:0 DeviceMinor:62 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/621ace500d4f75ef9c887619ca34581f60018c1b26a94ae2a4dcaf17ceb7fa74/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/09fceefd9630cf2a164a77f3bb0545dadde776dfacf0dc5e986919d2ee3116c5/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/62c6703912aecec773d95240640baeb4395beeab94b0ac8e1d379492343ac48c/userdata/shm DeviceMajor:0 DeviceMinor:81 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8a9cb48c43d1cf1ab491ef872fe88365a0da7a311fa8f4bb318400164ef97043/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2be5910f-12fa-46fe-bf9a-ddff510f4d09/volumes/kubernetes.io~projected/kube-api-access-2khhs DeviceMajor:0 DeviceMinor:216 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d31facfa-368e-4417-b9ce-30c3a495f650/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/757c56165d7a422d29db3395a4755e5bc5c7251c3dadac21fa6fb98b3fe8cca9/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/60e62efb-7c65-41f3-87d4-0805eec8f7e9/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:207 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dc2f830ae39d642b0c9b00c36a1f943093a33bb54659bc00bb35c1fd34a304e2/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a594de5b-9748-4355-86b8-95ca7dc9dda1/volumes/kubernetes.io~projected/kube-api-access-bkj5v DeviceMajor:0 DeviceMinor:63 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aba04aa0-2bee-47e2-9f36-8790309d6e06/volumes/kubernetes.io~projected/kube-api-access-xftjm DeviceMajor:0 DeviceMinor:217 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b80ffbfe-445f-431c-b8f4-be9fd2c5d094/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/10337f54-95e9-42cd-a73f-16817fc1cb96/volumes/kubernetes.io~projected/kube-api-access-p5nr5 DeviceMajor:0 DeviceMinor:221 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0/volumes/kubernetes.io~projected/kube-api-access-q6wnt DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bbb14793-e4b1-4912-9ce2-008cba7ce1d4/volumes/kubernetes.io~projected/kube-api-access-k2v9l DeviceMajor:0 DeviceMinor:213 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4d6911b025d9e88987f3642ba349b3c4e1559445b1f9f5408d99e1a9ed1e51a9/userdata/shm DeviceMajor:0 DeviceMinor:243 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fb8a16464c99266ad50f6c092a3fcb92ed6c2701b65333a20f9d5a63b1271d33/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4f13aca3-562e-406b-a154-8755b1bf7d15/volumes/kubernetes.io~projected/kube-api-access-cfh24 DeviceMajor:0 DeviceMinor:210 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d81730d3-53ed-40b8-a512-348fc042f821/volumes/kubernetes.io~projected/kube-api-access-sfwdk DeviceMajor:0 DeviceMinor:224 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-334 DeviceMajor:0 DeviceMinor:334 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/18fbde11-0050-4b55-bc23-a4e5e76ddc81/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bbb14793-e4b1-4912-9ce2-008cba7ce1d4/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:203 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cc28fe61-b116-4a8b-ad04-123d62365a02/volumes/kubernetes.io~projected/kube-api-access-2d7qp DeviceMajor:0 DeviceMinor:227 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3cd6393cdf0c12b4dadc6f89839a211107470255881edd2e736a1dfef317e7fd/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d31facfa-368e-4417-b9ce-30c3a495f650/volumes/kubernetes.io~projected/kube-api-access-8bm4p DeviceMajor:0 DeviceMinor:61 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-143 DeviceMajor:0 DeviceMinor:143 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/10a8a53c-d9d6-45aa-936e-48745236bb01/volumes/kubernetes.io~projected/kube-api-access-jknfr DeviceMajor:0 DeviceMinor:222 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/69652a499a9fef069224a36d8831b8831c15ee6dde86a929b4e48599cf0044f2/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9ed57772b7f777bf45d80ea1035ae4a8c4e3fc0876275655ceda617b1849ba7f/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d81730d3-53ed-40b8-a512-348fc042f821/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:205 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/79f0f110-a04f-4b35-8eba-c8f0469b62a1/volumes/kubernetes.io~projected/kube-api-access-bzdfs DeviceMajor:0 DeviceMinor:226 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/3ab05e61-6a22-4ced-abec-a528a15f9ba0/volumes/kubernetes.io~projected/kube-api-access-f86qq DeviceMajor:0 DeviceMinor:67 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-136 DeviceMajor:0 DeviceMinor:136 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4abb3ff147c2ca169b46c138813140b41dbe52d3eab89d6b4a2c31371d0197d7/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d/volumes/kubernetes.io~projected/kube-api-access-h54zx DeviceMajor:0 DeviceMinor:329 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/18fbde11-0050-4b55-bc23-a4e5e76ddc81/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/b80ffbfe-445f-431c-b8f4-be9fd2c5d094/volumes/kubernetes.io~projected/kube-api-access-9b6xs DeviceMajor:0 DeviceMinor:223 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d31facfa-368e-4417-b9ce-30c3a495f650/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cc28fe61-b116-4a8b-ad04-123d62365a02/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:225 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1263e83a-e0fd-4eb4-8141-5026aec6a631/volumes/kubernetes.io~projected/kube-api-access-825sw DeviceMajor:0 DeviceMinor:66 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/06910518-58db-450f-a5ee-521ac8f51e1a/volumes/kubernetes.io~projected/kube-api-access-slh7l DeviceMajor:0 DeviceMinor:211 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/60e62efb-7c65-41f3-87d4-0805eec8f7e9/volumes/kubernetes.io~projected/kube-api-access-xk2ml DeviceMajor:0 DeviceMinor:220 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/aa17ae3fe81ca3f5eb2127624190704b10c0c525c3427f5336204c20921ecb83/userdata/shm DeviceMajor:0 DeviceMinor:336 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e089d179-e583-4590-ba4c-39e7d486d707/volumes/kubernetes.io~projected/kube-api-access-rk26b DeviceMajor:0 DeviceMinor:219 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c88512809f317fb4be946430d40291f8d55c07f7231005bd8d88bd9496953990/userdata/shm DeviceMajor:0 DeviceMinor:294 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1af18b9b9ea3cf77550be3bc00adbc0fb260e197d40cdf8e54ab968dc6323fff/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ab1eba861addee696d5063000eae19b0fb3f0aa5c112d7ab33a2732fc5767e8e/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b4171e4a-4d6e-4901-983a-600ab76ada45/volumes/kubernetes.io~projected/kube-api-access-4ddnv DeviceMajor:0 DeviceMinor:327 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/918f40e9b236196597ce1d674a12bd818bd975806363262c6f485da82c1a5346/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a7e45249-4b17-415f-966d-af5321f182d5/volumes/kubernetes.io~projected/kube-api-access-lshc4 DeviceMajor:0 DeviceMinor:293 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d81730d3-53ed-40b8-a512-348fc042f821/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:202 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3a2f595c-6efe-4b69-81bf-074985a6ddde/volumes/kubernetes.io~projected/kube-api-access-fn8lm DeviceMajor:0 DeviceMinor:215 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/18fbde11-0050-4b55-bc23-a4e5e76ddc81/volumes/kubernetes.io~projected/kube-api-access-ppcp2 DeviceMajor:0 DeviceMinor:60 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bbb14793-e4b1-4912-9ce2-008cba7ce1d4/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/aba04aa0-2bee-47e2-9f36-8790309d6e06/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:204 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:1af18b9b9ea3cf7 MacAddress:52:1a:1b:95:bc:bd Speed:10000 Mtu:8901} {Name:3b7b2649a9fbcd9 MacAddress:1a:10:aa:65:a6:75 Speed:10000 Mtu:8901} {Name:4abb3ff147c2ca1 MacAddress:b2:5f:e1:55:18:87 Speed:10000 Mtu:8901} {Name:4d6911b025d9e88 MacAddress:fa:46:59:e7:76:1e Speed:10000 Mtu:8901} {Name:57296fb69fdd6aa MacAddress:16:9f:f6:92:c0:fc Speed:10000 Mtu:8901} {Name:69652a499a9fef0 MacAddress:1e:34:5e:a3:0c:d5 Speed:10000 Mtu:8901} {Name:918f40e9b236196 MacAddress:c2:02:8d:2f:b9:50 Speed:10000 Mtu:8901} {Name:966dcd2556e1ec1 MacAddress:e2:ee:cc:56:0a:3a Speed:10000 Mtu:8901} {Name:aa17ae3fe81ca3f MacAddress:26:4f:e4:d7:f5:53 Speed:10000 Mtu:8901} {Name:ab1eba861addee6 MacAddress:d2:ac:1b:10:82:26 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:e5:4d:34:e3 Speed:0 Mtu:9001} {Name:br-int MacAddress:36:d1:ed:8b:f1:73 Speed:0 Mtu:8901} {Name:c88512809f317fb MacAddress:3e:f3:03:72:f2:94 Speed:10000 Mtu:8901} {Name:dc2f830ae39d642 MacAddress:b6:1a:d5:4c:89:1d Speed:10000 Mtu:8901} {Name:e73ede3436e8237 MacAddress:5a:26:27:0b:c8:c3 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:e5:4d:34:e3 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:b6:61:17:9d:d0:67 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:ca:69:28:c3:7a:46 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.487581    4945 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.487638    4945 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.487980    4945 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.488003    4945 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-36.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.488193    4945 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.488206    4945 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.488224    4945 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.488242    4945 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.488268    4945 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.488478    4945 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.488654    4945 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.488706    4945 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.488720    4945 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.488735    4945 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.488746    4945 kubelet.go:405] "Adding apiserver pod source"
Oct 30 12:35:55.489782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.488758    4945 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 12:35:55.490793 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.490057    4945 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 12:35:55.490793 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.490072    4945 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 12:35:55.490793 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.490704    4945 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 12:35:55.493366 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493347    4945 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 12:35:55.493366 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493362    4945 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 12:35:55.493476 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493459    4945 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 12:35:55.493724 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493649    4945 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 12:35:55.493724 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493678    4945 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 12:35:55.493805 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493766    4945 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 12:35:55.493805 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493779    4945 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 12:35:55.493805 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493785    4945 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 12:35:55.493805 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493791    4945 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 12:35:55.493805 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493796    4945 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 12:35:55.493805 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493803    4945 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 12:35:55.493805 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493808    4945 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 12:35:55.494027 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493814    4945 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 12:35:55.494027 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493821    4945 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 12:35:55.494027 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493827    4945 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 12:35:55.494027 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493835    4945 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 12:35:55.494027 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493844    4945 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 12:35:55.494027 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493867    4945 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 12:35:55.494027 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.493874    4945 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 12:35:55.497285 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.497272    4945 server.go:1263] "Started kubelet"
Oct 30 12:35:55.497382 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.497350    4945 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 12:35:55.497435 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.497376    4945 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 12:35:55.497487 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.497456    4945 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 12:35:55.497836 ip-10-0-142-36 systemd[1]: Started Kubernetes Kubelet.
Oct 30 12:35:55.497982 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.497855    4945 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 12:35:55.498717 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.498702    4945 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.501941    4945 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.502633    4945 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.502645    4945 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.502691    4945 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.502739    4945 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.502739    4945 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 12:30:17 +0000 UTC" deadline="2028-04-10 11:47:51.773758226 +0000 UTC"
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.502770    4945 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12671h11m56.270991267s"
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.503481    4945 factory.go:55] Registering systemd factory
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.503501    4945 factory.go:223] Registration of the systemd container factory successfully
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.503845    4945 factory.go:153] Registering CRI-O factory
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.503867    4945 factory.go:223] Registration of the crio container factory successfully
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.503934    4945 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.503957    4945 factory.go:103] Registering Raw factory
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.503978    4945 manager.go:1199] Started watching for new ooms in manager
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.504591    4945 manager.go:319] Starting recovery of all containers
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.506734    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aba04aa0-2bee-47e2-9f36-8790309d6e06" volumeName="kubernetes.io/secret/aba04aa0-2bee-47e2-9f36-8790309d6e06-serving-cert" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.506772    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b80ffbfe-445f-431c-b8f4-be9fd2c5d094" volumeName="kubernetes.io/projected/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-kube-api-access-9b6xs" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.506796    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="18fbde11-0050-4b55-bc23-a4e5e76ddc81" volumeName="kubernetes.io/secret/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.506810    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60e62efb-7c65-41f3-87d4-0805eec8f7e9" volumeName="kubernetes.io/empty-dir/60e62efb-7c65-41f3-87d4-0805eec8f7e9-tmp" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.506829    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="79f0f110-a04f-4b35-8eba-c8f0469b62a1" volumeName="kubernetes.io/projected/79f0f110-a04f-4b35-8eba-c8f0469b62a1-kube-api-access-bzdfs" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.506843    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc28fe61-b116-4a8b-ad04-123d62365a02" volumeName="kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-trusted-ca" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.506857    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc28fe61-b116-4a8b-ad04-123d62365a02" volumeName="kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.506887    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d81730d3-53ed-40b8-a512-348fc042f821" volumeName="kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-default-certificate" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.506909    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1263e83a-e0fd-4eb4-8141-5026aec6a631" volumeName="kubernetes.io/configmap/1263e83a-e0fd-4eb4-8141-5026aec6a631-iptables-alerter-script" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.506930    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="15bb3274-f845-4851-90ae-f4ab08e685c8" volumeName="kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.506945    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4171e4a-4d6e-4901-983a-600ab76ada45" volumeName="kubernetes.io/projected/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-api-access-4ddnv" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.506958    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d31facfa-368e-4417-b9ce-30c3a495f650" volumeName="kubernetes.io/empty-dir/d31facfa-368e-4417-b9ce-30c3a495f650-etc-tuned" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.506977    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0" volumeName="kubernetes.io/empty-dir/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-tmp-dir" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507027    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b56b071-d626-4d8d-bba6-f5f69fa95db0" volumeName="kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507063    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce" volumeName="kubernetes.io/secret/7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce-agent-certs" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507105    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbb14793-e4b1-4912-9ce2-008cba7ce1d4" volumeName="kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507127    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b80ffbfe-445f-431c-b8f4-be9fd2c5d094" volumeName="kubernetes.io/secret/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-serving-cert" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507140    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbb14793-e4b1-4912-9ce2-008cba7ce1d4" volumeName="kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-trusted-ca" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507158    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbb14793-e4b1-4912-9ce2-008cba7ce1d4" volumeName="kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-kube-api-access-k2v9l" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507172    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10a8a53c-d9d6-45aa-936e-48745236bb01" volumeName="kubernetes.io/projected/10a8a53c-d9d6-45aa-936e-48745236bb01-kube-api-access-jknfr" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507183    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="18fbde11-0050-4b55-bc23-a4e5e76ddc81" volumeName="kubernetes.io/projected/18fbde11-0050-4b55-bc23-a4e5e76ddc81-kube-api-access-ppcp2" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507200    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b80ffbfe-445f-431c-b8f4-be9fd2c5d094" volumeName="kubernetes.io/configmap/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-config" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507213    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e089d179-e583-4590-ba4c-39e7d486d707" volumeName="kubernetes.io/empty-dir/e089d179-e583-4590-ba4c-39e7d486d707-tmp-dir" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507246    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="06910518-58db-450f-a5ee-521ac8f51e1a" volumeName="kubernetes.io/projected/06910518-58db-450f-a5ee-521ac8f51e1a-kube-api-access-slh7l" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507259    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ab05e61-6a22-4ced-abec-a528a15f9ba0" volumeName="kubernetes.io/configmap/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-daemon-config" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507271    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60e62efb-7c65-41f3-87d4-0805eec8f7e9" volumeName="kubernetes.io/configmap/60e62efb-7c65-41f3-87d4-0805eec8f7e9-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507288    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4171e4a-4d6e-4901-983a-600ab76ada45" volumeName="kubernetes.io/empty-dir/b4171e4a-4d6e-4901-983a-600ab76ada45-data-volume" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507299    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10337f54-95e9-42cd-a73f-16817fc1cb96" volumeName="kubernetes.io/secret/10337f54-95e9-42cd-a73f-16817fc1cb96-cert" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507325    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4f13aca3-562e-406b-a154-8755b1bf7d15" volumeName="kubernetes.io/secret/4f13aca3-562e-406b-a154-8755b1bf7d15-serving-cert" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507356    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbb14793-e4b1-4912-9ce2-008cba7ce1d4" volumeName="kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-certificates" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507369    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbb14793-e4b1-4912-9ce2-008cba7ce1d4" volumeName="kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-installation-pull-secrets" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507383    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b56b071-d626-4d8d-bba6-f5f69fa95db0" volumeName="kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507407    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4f13aca3-562e-406b-a154-8755b1bf7d15" volumeName="kubernetes.io/projected/4f13aca3-562e-406b-a154-8755b1bf7d15-kube-api-access-cfh24" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507420    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60e62efb-7c65-41f3-87d4-0805eec8f7e9" volumeName="kubernetes.io/projected/60e62efb-7c65-41f3-87d4-0805eec8f7e9-kube-api-access-xk2ml" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507453    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a7e45249-4b17-415f-966d-af5321f182d5" volumeName="kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507465    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d81730d3-53ed-40b8-a512-348fc042f821" volumeName="kubernetes.io/projected/d81730d3-53ed-40b8-a512-348fc042f821-kube-api-access-sfwdk" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507477    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e937e696-f801-42e7-9622-d774cca50028" volumeName="kubernetes.io/projected/e937e696-f801-42e7-9622-d774cca50028-kube-api-access-pmn28" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507494    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60e62efb-7c65-41f3-87d4-0805eec8f7e9" volumeName="kubernetes.io/empty-dir/60e62efb-7c65-41f3-87d4-0805eec8f7e9-snapshots" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507507    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc28fe61-b116-4a8b-ad04-123d62365a02" volumeName="kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-installation-pull-secrets" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507524    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e30d5797-e034-4806-be4f-54f54698920c" volumeName="kubernetes.io/projected/e30d5797-e034-4806-be4f-54f54698920c-kube-api-access-52p8s" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507538    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b56b071-d626-4d8d-bba6-f5f69fa95db0" volumeName="kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cni-binary-copy" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507550    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a594de5b-9748-4355-86b8-95ca7dc9dda1" volumeName="kubernetes.io/configmap/a594de5b-9748-4355-86b8-95ca7dc9dda1-serviceca" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507573    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aba04aa0-2bee-47e2-9f36-8790309d6e06" volumeName="kubernetes.io/configmap/aba04aa0-2bee-47e2-9f36-8790309d6e06-config" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507585    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aba04aa0-2bee-47e2-9f36-8790309d6e06" volumeName="kubernetes.io/projected/aba04aa0-2bee-47e2-9f36-8790309d6e06-kube-api-access-xftjm" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507653    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbb14793-e4b1-4912-9ce2-008cba7ce1d4" volumeName="kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-bound-sa-token" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507666    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2be5910f-12fa-46fe-bf9a-ddff510f4d09" volumeName="kubernetes.io/projected/2be5910f-12fa-46fe-bf9a-ddff510f4d09-kube-api-access-2khhs" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507680    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="336de054-d674-4f26-81d8-980b11e27ace" volumeName="kubernetes.io/projected/336de054-d674-4f26-81d8-980b11e27ace-kube-api-access-5fbjv" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507698    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3a2f595c-6efe-4b69-81bf-074985a6ddde" volumeName="kubernetes.io/projected/3a2f595c-6efe-4b69-81bf-074985a6ddde-kube-api-access-fn8lm" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507710    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bbb14793-e4b1-4912-9ce2-008cba7ce1d4" volumeName="kubernetes.io/empty-dir/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-ca-trust-extracted" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507735    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e089d179-e583-4590-ba4c-39e7d486d707" volumeName="kubernetes.io/projected/e089d179-e583-4590-ba4c-39e7d486d707-kube-api-access-rk26b" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507748    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="18fbde11-0050-4b55-bc23-a4e5e76ddc81" volumeName="kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-env-overrides" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507761    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2be5910f-12fa-46fe-bf9a-ddff510f4d09" volumeName="kubernetes.io/configmap/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-config" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507843    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49837f84-8643-4643-aa92-3716a9b755d1" volumeName="kubernetes.io/configmap/49837f84-8643-4643-aa92-3716a9b755d1-nginx-conf" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507860    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60e62efb-7c65-41f3-87d4-0805eec8f7e9" volumeName="kubernetes.io/secret/60e62efb-7c65-41f3-87d4-0805eec8f7e9-serving-cert" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507879    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d31facfa-368e-4417-b9ce-30c3a495f650" volumeName="kubernetes.io/projected/d31facfa-368e-4417-b9ce-30c3a495f650-kube-api-access-8bm4p" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507891    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d81730d3-53ed-40b8-a512-348fc042f821" volumeName="kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-stats-auth" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507956    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d" volumeName="kubernetes.io/projected/eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d-kube-api-access-h54zx" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.507974    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="18fbde11-0050-4b55-bc23-a4e5e76ddc81" volumeName="kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovnkube-config" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508020    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3a2f595c-6efe-4b69-81bf-074985a6ddde" volumeName="kubernetes.io/configmap/3a2f595c-6efe-4b69-81bf-074985a6ddde-telemetry-config" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508035    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ab05e61-6a22-4ced-abec-a528a15f9ba0" volumeName="kubernetes.io/projected/3ab05e61-6a22-4ced-abec-a528a15f9ba0-kube-api-access-f86qq" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508090    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4f13aca3-562e-406b-a154-8755b1bf7d15" volumeName="kubernetes.io/configmap/4f13aca3-562e-406b-a154-8755b1bf7d15-config" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508105    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce" volumeName="kubernetes.io/configmap/7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce-konnectivity-ca" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508156    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a594de5b-9748-4355-86b8-95ca7dc9dda1" volumeName="kubernetes.io/projected/a594de5b-9748-4355-86b8-95ca7dc9dda1-kube-api-access-bkj5v" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508169    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b80ffbfe-445f-431c-b8f4-be9fd2c5d094" volumeName="kubernetes.io/configmap/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-trusted-ca" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508214    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc28fe61-b116-4a8b-ad04-123d62365a02" volumeName="kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-kube-api-access-2d7qp" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508227    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10337f54-95e9-42cd-a73f-16817fc1cb96" volumeName="kubernetes.io/projected/10337f54-95e9-42cd-a73f-16817fc1cb96-kube-api-access-p5nr5" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508239    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1263e83a-e0fd-4eb4-8141-5026aec6a631" volumeName="kubernetes.io/projected/1263e83a-e0fd-4eb4-8141-5026aec6a631-kube-api-access-825sw" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508256    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc28fe61-b116-4a8b-ad04-123d62365a02" volumeName="kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-certificates" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508282    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc28fe61-b116-4a8b-ad04-123d62365a02" volumeName="kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-bound-sa-token" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508300    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d31facfa-368e-4417-b9ce-30c3a495f650" volumeName="kubernetes.io/empty-dir/d31facfa-368e-4417-b9ce-30c3a495f650-tmp" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508325    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e089d179-e583-4590-ba4c-39e7d486d707" volumeName="kubernetes.io/configmap/e089d179-e583-4590-ba4c-39e7d486d707-config-volume" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508352    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="18fbde11-0050-4b55-bc23-a4e5e76ddc81" volumeName="kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508383    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2758dbad-1f8a-4503-9dc2-6181518daeaf" volumeName="kubernetes.io/projected/2758dbad-1f8a-4503-9dc2-6181518daeaf-kube-api-access-q8xqw" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508395    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b56b071-d626-4d8d-bba6-f5f69fa95db0" volumeName="kubernetes.io/projected/4b56b071-d626-4d8d-bba6-f5f69fa95db0-kube-api-access-vhm96" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508468    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60e62efb-7c65-41f3-87d4-0805eec8f7e9" volumeName="kubernetes.io/configmap/60e62efb-7c65-41f3-87d4-0805eec8f7e9-service-ca-bundle" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508525    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc28fe61-b116-4a8b-ad04-123d62365a02" volumeName="kubernetes.io/empty-dir/cc28fe61-b116-4a8b-ad04-123d62365a02-ca-trust-extracted" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508551    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0" volumeName="kubernetes.io/projected/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-kube-api-access-q6wnt" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508568    4945 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ab05e61-6a22-4ced-abec-a528a15f9ba0" volumeName="kubernetes.io/configmap/3ab05e61-6a22-4ced-abec-a528a15f9ba0-cni-binary-copy" seLinuxMountContext=""
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508581    4945 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.508588    4945 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 12:35:55.508725 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:55.503954    4945 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-36.ec2.internal\" not found"
Oct 30 12:35:55.526325 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.526299    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/1.log"
Oct 30 12:35:55.526441 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.526403    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/1.log"
Oct 30 12:35:55.539706 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.539670    4945 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 12:35:55.541017 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.540999    4945 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 12:35:55.541017 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.541019    4945 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 12:35:55.541192 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.541038    4945 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 12:35:55.541192 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:55.541074    4945 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 12:35:55.552332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.552298    4945 generic.go:356] "Generic (PLEG): container finished" podID="45b9204b780ab5797493be0f231a6750" containerID="7d5cdf1ce80b70633707b99762a40ccac4b90fdc7a66be0683724f90c174a6c5" exitCode=0
Oct 30 12:35:55.557730 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.557708    4945 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:35:55.558112 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.558082    4945 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:35:55.560009 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.559846    4945 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 12:35:55.564942 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.564910    4945 generic.go:356] "Generic (PLEG): container finished" podID="4b56b071-d626-4d8d-bba6-f5f69fa95db0" containerID="c7b150ab472af683719dd2a5651cbb505b582e7e155eb5473f1fe723126e4d73" exitCode=0
Oct 30 12:35:55.564942 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.564939    4945 generic.go:356] "Generic (PLEG): container finished" podID="4b56b071-d626-4d8d-bba6-f5f69fa95db0" containerID="b86e9c179bdd5a1c6ff66dc564b9a6a9ad6b3ca46e3f9731f010bb2ef1e5964a" exitCode=0
Oct 30 12:35:55.565050 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.564951    4945 generic.go:356] "Generic (PLEG): container finished" podID="4b56b071-d626-4d8d-bba6-f5f69fa95db0" containerID="fa73653c08ffe682b691c171199ba34b881bcf0e65e66ef666ebfd4867a2f74b" exitCode=0
Oct 30 12:35:55.565050 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.564969    4945 generic.go:356] "Generic (PLEG): container finished" podID="4b56b071-d626-4d8d-bba6-f5f69fa95db0" containerID="57afecd87278bf3ee190f26209e7b0009092ffcf2e4ddff462c911e4a2a9d2b6" exitCode=0
Oct 30 12:35:55.565050 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.564978    4945 generic.go:356] "Generic (PLEG): container finished" podID="4b56b071-d626-4d8d-bba6-f5f69fa95db0" containerID="9727da6e4443a110df6503922683247d0818f78fdff3cb502b0d171519330a5e" exitCode=0
Oct 30 12:35:55.565050 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.564984    4945 generic.go:356] "Generic (PLEG): container finished" podID="4b56b071-d626-4d8d-bba6-f5f69fa95db0" containerID="31ce6209c88915a87e9720f4ce8a2b0483c3d61710c08081de978b881fc89a5a" exitCode=0
Oct 30 12:35:55.569705 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.569685    4945 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:35:55.569800 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.569719    4945 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 12:35:55.577335 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.577318    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/1.log"
Oct 30 12:35:55.577419 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.577341    4945 generic.go:356] "Generic (PLEG): container finished" podID="b80ffbfe-445f-431c-b8f4-be9fd2c5d094" containerID="962e40dd4a1466cf55a7d8df3804586fbec551f4fb8919a94013f65991c6296b" exitCode=255
Oct 30 12:35:55.605143 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.605090    4945 manager.go:324] Recovery completed
Oct 30 12:35:55.625220 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.625201    4945 cpu_manager.go:226] "Starting" policy="none"
Oct 30 12:35:55.625220 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.625216    4945 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 12:35:55.625344 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.625232    4945 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 12:35:55.625374 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.625343    4945 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 12:35:55.625374 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.625353    4945 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 12:35:55.625374 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.625369    4945 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 12:35:55.625374 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.625374    4945 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 12:35:55.625528 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.625379    4945 policy_none.go:50] "Start"
Oct 30 12:35:55.625528 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.625396    4945 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 12:35:55.625528 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.625404    4945 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 12:35:55.625528 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.625524    4945 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 12:35:55.625654 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.625531    4945 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 12:35:55.625654 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.625536    4945 policy_none.go:44] "Start"
Oct 30 12:35:55.629781 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.629758    4945 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 12:35:55.632363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.632349    4945 manager.go:341] "Starting Device Plugin manager"
Oct 30 12:35:55.632877 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:55.632384    4945 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 12:35:55.632917 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.632881    4945 server.go:87] "Starting device plugin registration server"
Oct 30 12:35:55.633130 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.633120    4945 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 12:35:55.633161 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.633132    4945 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 12:35:55.633260 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.633237    4945 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 12:35:55.633396 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.633352    4945 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 12:35:55.633507 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.633408    4945 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 12:35:55.633507 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.633416    4945 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 12:35:55.633626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.633468    4945 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T12:35:55.633391438Z","UUID":"29d5298d-8438-49df-9e74-dcfe9b6c21d0","Handler":null,"Name":"","Endpoint":""}
Oct 30 12:35:55.634821 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:55.634797    4945 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 12:35:55.634951 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.634935    4945 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 12:35:55.635012 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.634984    4945 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 12:35:55.641150 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.641120    4945 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal"]
Oct 30 12:35:55.641766 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.641724    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal" event={"ID":"45b9204b780ab5797493be0f231a6750","Type":"ContainerStarted","Data":"96909b7980c00dee51ef2280b27268d3dea44ed88dd69a42df305f4aaf6f2346"}
Oct 30 12:35:55.641766 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.641762    4945 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.641905 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.641784    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal" event={"ID":"45b9204b780ab5797493be0f231a6750","Type":"ContainerDied","Data":"7d5cdf1ce80b70633707b99762a40ccac4b90fdc7a66be0683724f90c174a6c5"}
Oct 30 12:35:55.641905 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.641807    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal" event={"ID":"45b9204b780ab5797493be0f231a6750","Type":"ContainerStarted","Data":"fb8a16464c99266ad50f6c092a3fcb92ed6c2701b65333a20f9d5a63b1271d33"}
Oct 30 12:35:55.641905 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.641825    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal" event={"ID":"6766e4bd763f65c3c761bb65d5e5c301","Type":"ContainerStarted","Data":"6a351f52b540041bd424a6987c8fd0328f5be95f20254374632ac74f48d39427"}
Oct 30 12:35:55.641905 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.641828    4945 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.642220 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.641842    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal" event={"ID":"6766e4bd763f65c3c761bb65d5e5c301","Type":"ContainerStarted","Data":"c2ce37edc542d23f0715646c0c806e458b1cf6afdb37838f35b00e122f08fb4a"}
Oct 30 12:35:55.668050 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.668034    4945 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:35:55.668128 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:55.668081    4945 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-36.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.668588 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.668576    4945 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:35:55.668719 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:55.668622    4945 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.733355 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.733334    4945 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 12:35:55.734649 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.734633    4945 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 12:35:55.734719 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.734660    4945 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 12:35:55.734719 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.734669    4945 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-36.ec2.internal" event="NodeHasSufficientPID"
Oct 30 12:35:55.734719 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.734705    4945 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.760257 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.760237    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45b9204b780ab5797493be0f231a6750-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal\" (UID: \"45b9204b780ab5797493be0f231a6750\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.760318 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.760264    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45b9204b780ab5797493be0f231a6750-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal\" (UID: \"45b9204b780ab5797493be0f231a6750\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.760318 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.760282    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6766e4bd763f65c3c761bb65d5e5c301-config\") pod \"kube-apiserver-proxy-ip-10-0-142-36.ec2.internal\" (UID: \"6766e4bd763f65c3c761bb65d5e5c301\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.767568 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.767551    4945 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.767645 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.767623    4945 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.860995 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.860932    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45b9204b780ab5797493be0f231a6750-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal\" (UID: \"45b9204b780ab5797493be0f231a6750\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.860995 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.860888    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/45b9204b780ab5797493be0f231a6750-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal\" (UID: \"45b9204b780ab5797493be0f231a6750\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.861125 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.860995    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45b9204b780ab5797493be0f231a6750-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal\" (UID: \"45b9204b780ab5797493be0f231a6750\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.861125 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.861043    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/45b9204b780ab5797493be0f231a6750-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal\" (UID: \"45b9204b780ab5797493be0f231a6750\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.861125 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.861052    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6766e4bd763f65c3c761bb65d5e5c301-config\") pod \"kube-apiserver-proxy-ip-10-0-142-36.ec2.internal\" (UID: \"6766e4bd763f65c3c761bb65d5e5c301\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:55.861125 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:55.861077    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6766e4bd763f65c3c761bb65d5e5c301-config\") pod \"kube-apiserver-proxy-ip-10-0-142-36.ec2.internal\" (UID: \"6766e4bd763f65c3c761bb65d5e5c301\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:56.489651 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.489623    4945 apiserver.go:52] "Watching apiserver"
Oct 30 12:35:56.563316 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.563285    4945 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 12:35:56.563973 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.563950    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lw4gw","kube-system/kas-connection-checker-685895c7c9-5lwxr","openshift-insights/insights-operator-7c4986d449-m6xzg","openshift-ingress-canary/ingress-canary-t5j98","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w","kube-system/global-pull-secret-syncer-ntzbv","openshift-multus/multus-additional-cni-plugins-cpq89","openshift-multus/network-metrics-daemon-xdcfv","openshift-ingress/router-default-846cf5b65-gllvj","kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal","openshift-network-diagnostics/network-check-target-29mpd","openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k","kube-system/kas-connection-checker-685895c7c9-q8mz6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal","kube-system/konnectivity-agent-p2z49","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6mt4g","openshift-image-registry/node-ca-pt7kp","openshift-kube-storage-version-migrator/migrator-66df5d496b-tjlfm","openshift-ovn-kubernetes/ovnkube-node-jk5qv","openshift-multus/multus-kqrtg","openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j","openshift-network-console/networking-console-plugin-86c478bb88-44v66","openshift-dns/node-resolver-cwcj7","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44","openshift-image-registry/image-registry-6f558d755b-bj4vz","openshift-console-operator/console-operator-7cd56b8dbc-8gvm8","openshift-image-registry/image-registry-57b7896669-ndwsr","openshift-dns/dns-default-2djq9","openshift-cluster-node-tuning-operator/tuned-5frn2","openshift-network-operator/iptables-alerter-k9987","kube-system/kas-connection-checker-685895c7c9-q74lv","openshift-network-diagnostics/network-check-source-6f78d59f99-gzkb8"]
Oct 30 12:35:56.565246 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.565225    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:56.569577 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.569552    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:56.570250 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.570234    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:56.570970 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.570953    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.571353 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.571334    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:56.575169 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.575152    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.577269 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.577250    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:56.578508 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.578481    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:56.578925 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.578898    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.579002 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.578932    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.579307 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.579290    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:56.579684 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.579658    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.580644 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.580232    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 12:35:56.580644 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.580241    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 12:35:56.580644 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.580279    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hpqh5\""
Oct 30 12:35:56.580644 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.580269    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.580644 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.580363    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-nnzn8\""
Oct 30 12:35:56.580644 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.580545    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.581002 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.580810    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.581049 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.581023    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qgn2s\""
Oct 30 12:35:56.581049 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.581041    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 12:35:56.581140 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.581083    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.581185 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.581154    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.581306 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.581289    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 12:35:56.582627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.582593    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9zjv6\""
Oct 30 12:35:56.583016 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.582998    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 12:35:56.583071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.583017    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.583129 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.583110    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.583190 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.583178    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 12:35:56.583404 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.583388    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 12:35:56.583506 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.583494    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.583632 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.583618    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 12:35:56.583756 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.583738    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.583833 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.583820    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 12:35:56.594997 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.594941    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-c8dt7\""
Oct 30 12:35:56.595615 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595585    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.595681 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595592    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.595681 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595671    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 12:35:56.595795 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595672    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.595795 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595722    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.595795 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595752    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 12:35:56.595795 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595756    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.595795 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595787    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 12:35:56.596006 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595779    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-hqrrz\""
Oct 30 12:35:56.596006 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595838    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-t6s7j\""
Oct 30 12:35:56.596006 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595847    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 12:35:56.596006 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595869    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-vwpvk\""
Oct 30 12:35:56.596006 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595878    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 12:35:56.596006 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595930    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 12:35:56.596006 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595934    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 12:35:56.596006 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.595976    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.596006 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.596007    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 12:35:56.596433 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.596047    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 12:35:56.596433 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.596178    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vtqg8\""
Oct 30 12:35:56.596433 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.596210    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 12:35:56.596433 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.596226    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 12:35:56.596433 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.596250    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.596433 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.596261    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.596433 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.596278    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 12:35:56.596433 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.596283    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2vwzh\""
Oct 30 12:35:56.596433 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.596344    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-8r6jz\""
Oct 30 12:35:56.596433 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.596385    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-8phcg\""
Oct 30 12:35:56.596433 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.596392    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.596855 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.596695    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 12:35:56.598771 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.598756    4945 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:56.598918 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.598906    4945 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:56.603272 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.603255    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 12:35:56.603877 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.603859    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.603877 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.603868    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 12:35:56.603998 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.603875    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.604914 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.604899    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.605000 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.604919    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 12:35:56.605065 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605046    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.605065 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605059    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.605187 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605055    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 12:35:56.605187 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605066    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 12:35:56.605187 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605103    4945 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 12:35:56.605187 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605113    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 12:35:56.605187 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605119    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.605187 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605153    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 12:35:56.605379 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605211    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.605379 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605221    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.605379 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605228    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 12:35:56.605379 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605242    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.605379 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605248    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.605379 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605256    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 12:35:56.605379 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605212    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.605379 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605250    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 12:35:56.605379 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605228    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.605379 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605222    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.605652 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.605387    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 12:35:56.606588 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.606566    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 12:35:56.606671 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.606588    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 12:35:56.606671 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.606650    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.606671 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.606655    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.606768 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.606673    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 12:35:56.606768 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.606691    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 12:35:56.606855 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.606813    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 12:35:56.606931 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.606919    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-562t6\""
Oct 30 12:35:56.606986 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.606918    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rkrpr\""
Oct 30 12:35:56.606986 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.606960    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 12:35:56.607073 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.606988    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-x6m2k\""
Oct 30 12:35:56.607073 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607008    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 12:35:56.607073 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607033    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-pdckv\""
Oct 30 12:35:56.607206 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607076    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gmdlh\""
Oct 30 12:35:56.607206 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607077    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kkflg\""
Oct 30 12:35:56.607206 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607128    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-54vc4\""
Oct 30 12:35:56.607206 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607193    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-bg2k7\""
Oct 30 12:35:56.607206 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607199    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-scxms\""
Oct 30 12:35:56.607478 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607444    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 12:35:56.607478 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607445    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 12:35:56.607478 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607471    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 12:35:56.607635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607481    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 12:35:56.607635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607464    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 12:35:56.607635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607482    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t86w7\""
Oct 30 12:35:56.607635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607459    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-6qrlt\""
Oct 30 12:35:56.607635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607520    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2mbkm\""
Oct 30 12:35:56.607854 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607707    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-x7vtx\""
Oct 30 12:35:56.607854 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607777    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 12:35:56.607854 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607844    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qjz2q\""
Oct 30 12:35:56.607992 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.607908    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tz7hb\""
Oct 30 12:35:56.621984 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.621963    4945 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:35:56.622066 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.622007    4945 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-36.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:56.622400 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.622384    4945 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 12:35:56.622457 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.622415    4945 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-36.ec2.internal"
Oct 30 12:35:56.664055 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664034    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk2ml\" (UniqueName: \"kubernetes.io/projected/60e62efb-7c65-41f3-87d4-0805eec8f7e9-kube-api-access-xk2ml\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:56.664140 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664065    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-serving-cert\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:56.664140 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664089    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fbjv\" (UniqueName: \"kubernetes.io/projected/336de054-d674-4f26-81d8-980b11e27ace-kube-api-access-5fbjv\") pod \"volume-data-source-validator-58c9f5f46d-6mt4g\" (UID: \"336de054-d674-4f26-81d8-980b11e27ace\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6mt4g"
Oct 30 12:35:56.664140 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664113    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ab05e61-6a22-4ced-abec-a528a15f9ba0-cni-binary-copy\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.664301 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664141    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfwdk\" (UniqueName: \"kubernetes.io/projected/d81730d3-53ed-40b8-a512-348fc042f821-kube-api-access-sfwdk\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:56.664301 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664170    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-slash\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.664301 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664196    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/15bb3274-f845-4851-90ae-f4ab08e685c8-kubelet-config\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:56.664301 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664222    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5nr5\" (UniqueName: \"kubernetes.io/projected/10337f54-95e9-42cd-a73f-16817fc1cb96-kube-api-access-p5nr5\") pod \"ingress-canary-t5j98\" (UID: \"10337f54-95e9-42cd-a73f-16817fc1cb96\") " pod="openshift-ingress-canary/ingress-canary-t5j98"
Oct 30 12:35:56.664301 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664296    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-serving-cert\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:56.664542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664347    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-cni-bin\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.664542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664377    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-modprobe-d\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.664542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664425    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d31facfa-368e-4417-b9ce-30c3a495f650-etc-tuned\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.664542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664453    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-ca-trust-extracted\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.664542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664468    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3ab05e61-6a22-4ced-abec-a528a15f9ba0-cni-binary-copy\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.664542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664500    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-tmp-dir\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:56.664542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664520    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d31facfa-368e-4417-b9ce-30c3a495f650-etc-tuned\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.664542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664537    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-systemd-units\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.664921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664552    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-ca-trust-extracted\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.664921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664567    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4171e4a-4d6e-4901-983a-600ab76ada45-data-volume\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:56.664921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664586    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-tmp-dir\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:56.664921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664593    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce-konnectivity-ca\") pod \"konnectivity-agent-p2z49\" (UID: \"7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce\") " pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:56.664921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664648    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6wnt\" (UniqueName: \"kubernetes.io/projected/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-kube-api-access-q6wnt\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:56.664921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664663    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b4171e4a-4d6e-4901-983a-600ab76ada45-data-volume\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:56.664921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664668    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lshc4\" (UniqueName: \"kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4\") pod \"network-check-target-29mpd\" (UID: \"a7e45249-4b17-415f-966d-af5321f182d5\") " pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:56.664921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664712    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-bound-sa-token\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.664921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664740    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.664921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664767    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhm96\" (UniqueName: \"kubernetes.io/projected/4b56b071-d626-4d8d-bba6-f5f69fa95db0-kube-api-access-vhm96\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.664921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664833    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-stats-auth\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:56.664921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664859    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-var-lib-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.664921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664880    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce-konnectivity-ca\") pod \"konnectivity-agent-p2z49\" (UID: \"7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce\") " pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:56.664921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664885    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-config\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:56.664921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664911    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-trusted-ca\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664937    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-image-registry-private-configuration\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664964    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-multus-certs\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.664990    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665057    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665081    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52p8s\" (UniqueName: \"kubernetes.io/projected/e30d5797-e034-4806-be4f-54f54698920c-kube-api-access-52p8s\") pod \"kas-connection-checker-685895c7c9-5lwxr\" (UID: \"e30d5797-e034-4806-be4f-54f54698920c\") " pod="kube-system/kas-connection-checker-685895c7c9-5lwxr"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665084    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-config\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665084    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-stats-auth\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665103    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665161    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-image-registry-private-configuration\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665240    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-trusted-ca\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665282    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aba04aa0-2bee-47e2-9f36-8790309d6e06-config\") pod \"service-ca-operator-544d749c8c-x976j\" (UID: \"aba04aa0-2bee-47e2-9f36-8790309d6e06\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665319    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-installation-pull-secrets\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665348    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-sys\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665372    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1263e83a-e0fd-4eb4-8141-5026aec6a631-host-slash\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665401    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aba04aa0-2bee-47e2-9f36-8790309d6e06-serving-cert\") pod \"service-ca-operator-544d749c8c-x976j\" (UID: \"aba04aa0-2bee-47e2-9f36-8790309d6e06\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:56.665434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665427    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xftjm\" (UniqueName: \"kubernetes.io/projected/aba04aa0-2bee-47e2-9f36-8790309d6e06-kube-api-access-xftjm\") pod \"service-ca-operator-544d749c8c-x976j\" (UID: \"aba04aa0-2bee-47e2-9f36-8790309d6e06\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665442    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665452    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665453    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/aba04aa0-2bee-47e2-9f36-8790309d6e06-config\") pod \"service-ca-operator-544d749c8c-x976j\" (UID: \"aba04aa0-2bee-47e2-9f36-8790309d6e06\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665478    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665519    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk26b\" (UniqueName: \"kubernetes.io/projected/e089d179-e583-4590-ba4c-39e7d486d707-kube-api-access-rk26b\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665555    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/aba04aa0-2bee-47e2-9f36-8790309d6e06-serving-cert\") pod \"service-ca-operator-544d749c8c-x976j\" (UID: \"aba04aa0-2bee-47e2-9f36-8790309d6e06\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665568    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-installation-pull-secrets\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665582    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-etc-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665712    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovn-node-metrics-cert\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665741    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cnibin\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665780    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-systemd\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665813    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665834    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60e62efb-7c65-41f3-87d4-0805eec8f7e9-tmp\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665863    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-etc-kubernetes\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665914    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60e62efb-7c65-41f3-87d4-0805eec8f7e9-tmp\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665961    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e089d179-e583-4590-ba4c-39e7d486d707-config-volume\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.665970    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovn-node-metrics-cert\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.666047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666021    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666059    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-socket-dir-parent\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666116    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-kubelet\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666139    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e089d179-e583-4590-ba4c-39e7d486d707-config-volume\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666150    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-daemon-config\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666179    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn8lm\" (UniqueName: \"kubernetes.io/projected/3a2f595c-6efe-4b69-81bf-074985a6ddde-kube-api-access-fn8lm\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666205    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/49837f84-8643-4643-aa92-3716a9b755d1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666229    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-cni-bin\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666276    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc28fe61-b116-4a8b-ad04-123d62365a02-ca-trust-extracted\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666301    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-kubernetes\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666326    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8xqw\" (UniqueName: \"kubernetes.io/projected/2758dbad-1f8a-4503-9dc2-6181518daeaf-kube-api-access-q8xqw\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666350    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-kubelet\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666378    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-log-socket\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666388    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-daemon-config\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666404    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovnkube-config\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666408    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc28fe61-b116-4a8b-ad04-123d62365a02-ca-trust-extracted\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666422    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/49837f84-8643-4643-aa92-3716a9b755d1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666429    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-cnibin\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666455    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f86qq\" (UniqueName: \"kubernetes.io/projected/3ab05e61-6a22-4ced-abec-a528a15f9ba0-kube-api-access-f86qq\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666481    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2khhs\" (UniqueName: \"kubernetes.io/projected/2be5910f-12fa-46fe-bf9a-ddff510f4d09-kube-api-access-2khhs\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666542    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cni-binary-copy\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666569    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysctl-conf\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666571    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovnkube-config\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666616    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-bound-sa-token\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666643    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce-agent-certs\") pod \"konnectivity-agent-p2z49\" (UID: \"7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce\") " pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666667    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:56.666738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666708    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkj5v\" (UniqueName: \"kubernetes.io/projected/a594de5b-9748-4355-86b8-95ca7dc9dda1-kube-api-access-bkj5v\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:56.667513 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666829    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/15bb3274-f845-4851-90ae-f4ab08e685c8-dbus\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:56.667513 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666864    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b4171e4a-4d6e-4901-983a-600ab76ada45-crio-socket\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:56.667513 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666893    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cni-binary-copy\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.667513 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666899    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-trusted-ca\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.667513 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666913    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce-agent-certs\") pod \"konnectivity-agent-p2z49\" (UID: \"7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce\") " pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:35:56.667513 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666924    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-run-netns\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.667513 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.666974    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d7qp\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-kube-api-access-2d7qp\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.667513 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.667006    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-cni-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.667513 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.667031    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:56.667513 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.667077    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-trusted-ca\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.667513 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.667135    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-825sw\" (UniqueName: \"kubernetes.io/projected/1263e83a-e0fd-4eb4-8141-5026aec6a631-kube-api-access-825sw\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:56.667513 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.667200    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-trusted-ca\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.668092 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.667678    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-env-overrides\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.668092 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.667739    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-registration-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.668092 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.667850    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-cni-multus\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.668092 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.667912    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-system-cni-dir\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.668092 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.667984    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e62efb-7c65-41f3-87d4-0805eec8f7e9-service-ca-bundle\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:56.668336 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.668099    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b6xs\" (UniqueName: \"kubernetes.io/projected/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-kube-api-access-9b6xs\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:56.668336 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.668142    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-k8s-cni-cncf-io\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.668336 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.668241    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2v9l\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-kube-api-access-k2v9l\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.668336 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.668272    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:56.668336 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.668314    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-env-overrides\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.668336 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.668322    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-sys-fs\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.668621 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.668360    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10337f54-95e9-42cd-a73f-16817fc1cb96-cert\") pod \"ingress-canary-t5j98\" (UID: \"10337f54-95e9-42cd-a73f-16817fc1cb96\") " pod="openshift-ingress-canary/ingress-canary-t5j98"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.668872    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-trusted-ca\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.668998    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d31facfa-368e-4417-b9ce-30c3a495f650-tmp\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669060    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzdfs\" (UniqueName: \"kubernetes.io/projected/79f0f110-a04f-4b35-8eba-c8f0469b62a1-kube-api-access-bzdfs\") pod \"kas-connection-checker-685895c7c9-q8mz6\" (UID: \"79f0f110-a04f-4b35-8eba-c8f0469b62a1\") " pod="kube-system/kas-connection-checker-685895c7c9-q8mz6"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669112    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-node-log\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669361    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669404    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/60e62efb-7c65-41f3-87d4-0805eec8f7e9-snapshots\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669412    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10337f54-95e9-42cd-a73f-16817fc1cb96-cert\") pod \"ingress-canary-t5j98\" (UID: \"10337f54-95e9-42cd-a73f-16817fc1cb96\") " pod="openshift-ingress-canary/ingress-canary-t5j98"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669443    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669528    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d31facfa-368e-4417-b9ce-30c3a495f650-tmp\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669584    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bm4p\" (UniqueName: \"kubernetes.io/projected/d31facfa-368e-4417-b9ce-30c3a495f650-kube-api-access-8bm4p\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669643    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-installation-pull-secrets\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669691    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669735    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1263e83a-e0fd-4eb4-8141-5026aec6a631-iptables-alerter-script\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669770    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669801    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-system-cni-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669892    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-conf-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669927    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/60e62efb-7c65-41f3-87d4-0805eec8f7e9-serving-cert\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669935    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-installation-pull-secrets\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669980    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-device-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.670023    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3a2f595c-6efe-4b69-81bf-074985a6ddde-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.670031    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/60e62efb-7c65-41f3-87d4-0805eec8f7e9-snapshots\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.670076    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-netns\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.670122    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.670173    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.670245    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfh24\" (UniqueName: \"kubernetes.io/projected/4f13aca3-562e-406b-a154-8755b1bf7d15-kube-api-access-cfh24\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w\" (UID: \"4f13aca3-562e-406b-a154-8755b1bf7d15\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.670284    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-run-ovn-kubernetes\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.670297    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1263e83a-e0fd-4eb4-8141-5026aec6a631-iptables-alerter-script\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.670324    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.670357    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysconfig\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.669072    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e62efb-7c65-41f3-87d4-0805eec8f7e9-service-ca-bundle\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:56.670626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.670523    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/60e62efb-7c65-41f3-87d4-0805eec8f7e9-serving-cert\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.670779    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/15bb3274-f845-4851-90ae-f4ab08e685c8-original-pull-secret\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.670796    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3a2f595c-6efe-4b69-81bf-074985a6ddde-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.671045    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-run\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.671132    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-certificates\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.671166    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-default-certificate\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.671203    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-ovn\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.671392    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-certificates\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.671444    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jknfr\" (UniqueName: \"kubernetes.io/projected/10a8a53c-d9d6-45aa-936e-48745236bb01-kube-api-access-jknfr\") pod \"network-check-source-6f78d59f99-gzkb8\" (UID: \"10a8a53c-d9d6-45aa-936e-48745236bb01\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-gzkb8"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.671481    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-var-lib-kubelet\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.671518    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a594de5b-9748-4355-86b8-95ca7dc9dda1-serviceca\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.671615    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppcp2\" (UniqueName: \"kubernetes.io/projected/18fbde11-0050-4b55-bc23-a4e5e76ddc81-kube-api-access-ppcp2\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.671650    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slh7l\" (UniqueName: \"kubernetes.io/projected/06910518-58db-450f-a5ee-521ac8f51e1a-kube-api-access-slh7l\") pod \"kas-connection-checker-685895c7c9-q74lv\" (UID: \"06910518-58db-450f-a5ee-521ac8f51e1a\") " pod="kube-system/kas-connection-checker-685895c7c9-q74lv"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.671683    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-socket-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.671727    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-etc-selinux\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.671756    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmn28\" (UniqueName: \"kubernetes.io/projected/e937e696-f801-42e7-9622-d774cca50028-kube-api-access-pmn28\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.671955    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a594de5b-9748-4355-86b8-95ca7dc9dda1-serviceca\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.672181    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-default-certificate\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:56.672332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.672313    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f13aca3-562e-406b-a154-8755b1bf7d15-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w\" (UID: \"4f13aca3-562e-406b-a154-8755b1bf7d15\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:56.673060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.672357    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-hosts-file\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:56.673060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.672472    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:56.673060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.672722    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ddnv\" (UniqueName: \"kubernetes.io/projected/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-api-access-4ddnv\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:56.673060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.672775    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-image-registry-private-configuration\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.673060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.672814    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-hostroot\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.673060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.672869    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-systemd\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.673060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.672907    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-lib-modules\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.673060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.672971    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f13aca3-562e-406b-a154-8755b1bf7d15-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w\" (UID: \"4f13aca3-562e-406b-a154-8755b1bf7d15\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:56.673060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673006    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4f13aca3-562e-406b-a154-8755b1bf7d15-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w\" (UID: \"4f13aca3-562e-406b-a154-8755b1bf7d15\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:56.673060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673039    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.673402 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673095    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-image-registry-private-configuration\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.673402 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673111    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysctl-d\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.673402 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673199    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4f13aca3-562e-406b-a154-8755b1bf7d15-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w\" (UID: \"4f13aca3-562e-406b-a154-8755b1bf7d15\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:56.673624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673584    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-certificates\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.673677 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673590    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4b56b071-d626-4d8d-bba6-f5f69fa95db0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.673677 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673643    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-cni-netd\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.673677 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673661    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-host\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.673820 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673679    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54zx\" (UniqueName: \"kubernetes.io/projected/eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d-kube-api-access-h54zx\") pod \"migrator-66df5d496b-tjlfm\" (UID: \"eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tjlfm"
Oct 30 12:35:56.673820 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673695    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e62efb-7c65-41f3-87d4-0805eec8f7e9-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:56.673820 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673711    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e089d179-e583-4590-ba4c-39e7d486d707-tmp-dir\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:56.673820 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673781    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-certificates\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.673820 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673789    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e089d179-e583-4590-ba4c-39e7d486d707-tmp-dir\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:56.673972 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673827    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a594de5b-9748-4355-86b8-95ca7dc9dda1-host\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:56.673972 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.673869    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovnkube-script-lib\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.674046 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.674029    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60e62efb-7c65-41f3-87d4-0805eec8f7e9-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:56.674097 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.674075    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/18fbde11-0050-4b55-bc23-a4e5e76ddc81-ovnkube-script-lib\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.681219 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.681197    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fbjv\" (UniqueName: \"kubernetes.io/projected/336de054-d674-4f26-81d8-980b11e27ace-kube-api-access-5fbjv\") pod \"volume-data-source-validator-58c9f5f46d-6mt4g\" (UID: \"336de054-d674-4f26-81d8-980b11e27ace\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-6mt4g"
Oct 30 12:35:56.683519 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.683499    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfwdk\" (UniqueName: \"kubernetes.io/projected/d81730d3-53ed-40b8-a512-348fc042f821-kube-api-access-sfwdk\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:56.685862 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.685843    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xftjm\" (UniqueName: \"kubernetes.io/projected/aba04aa0-2bee-47e2-9f36-8790309d6e06-kube-api-access-xftjm\") pod \"service-ca-operator-544d749c8c-x976j\" (UID: \"aba04aa0-2bee-47e2-9f36-8790309d6e06\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j"
Oct 30 12:35:56.686158 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.686130    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52p8s\" (UniqueName: \"kubernetes.io/projected/e30d5797-e034-4806-be4f-54f54698920c-kube-api-access-52p8s\") pod \"kas-connection-checker-685895c7c9-5lwxr\" (UID: \"e30d5797-e034-4806-be4f-54f54698920c\") " pod="kube-system/kas-connection-checker-685895c7c9-5lwxr"
Oct 30 12:35:56.686995 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.686593    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6wnt\" (UniqueName: \"kubernetes.io/projected/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-kube-api-access-q6wnt\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:56.686995 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.686929    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-bound-sa-token\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.686995 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.686949    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lshc4\" (UniqueName: \"kubernetes.io/projected/a7e45249-4b17-415f-966d-af5321f182d5-kube-api-access-lshc4\") pod \"network-check-target-29mpd\" (UID: \"a7e45249-4b17-415f-966d-af5321f182d5\") " pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:35:56.686995 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.686985    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2khhs\" (UniqueName: \"kubernetes.io/projected/2be5910f-12fa-46fe-bf9a-ddff510f4d09-kube-api-access-2khhs\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:56.687183 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.686985    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-825sw\" (UniqueName: \"kubernetes.io/projected/1263e83a-e0fd-4eb4-8141-5026aec6a631-kube-api-access-825sw\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:56.687183 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.687065    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhm96\" (UniqueName: \"kubernetes.io/projected/4b56b071-d626-4d8d-bba6-f5f69fa95db0-kube-api-access-vhm96\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.687183 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.687076    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk2ml\" (UniqueName: \"kubernetes.io/projected/60e62efb-7c65-41f3-87d4-0805eec8f7e9-kube-api-access-xk2ml\") pod \"insights-operator-7c4986d449-m6xzg\" (UID: \"60e62efb-7c65-41f3-87d4-0805eec8f7e9\") " pod="openshift-insights/insights-operator-7c4986d449-m6xzg"
Oct 30 12:35:56.688574 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.688556    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn8lm\" (UniqueName: \"kubernetes.io/projected/3a2f595c-6efe-4b69-81bf-074985a6ddde-kube-api-access-fn8lm\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:56.688650 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.688634    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f86qq\" (UniqueName: \"kubernetes.io/projected/3ab05e61-6a22-4ced-abec-a528a15f9ba0-kube-api-access-f86qq\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.689042 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.689027    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b6xs\" (UniqueName: \"kubernetes.io/projected/b80ffbfe-445f-431c-b8f4-be9fd2c5d094-kube-api-access-9b6xs\") pod \"console-operator-7cd56b8dbc-8gvm8\" (UID: \"b80ffbfe-445f-431c-b8f4-be9fd2c5d094\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:35:56.689448 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.689427    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5nr5\" (UniqueName: \"kubernetes.io/projected/10337f54-95e9-42cd-a73f-16817fc1cb96-kube-api-access-p5nr5\") pod \"ingress-canary-t5j98\" (UID: \"10337f54-95e9-42cd-a73f-16817fc1cb96\") " pod="openshift-ingress-canary/ingress-canary-t5j98"
Oct 30 12:35:56.689640 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.689626    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d7qp\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-kube-api-access-2d7qp\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.691159 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.691138    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-bound-sa-token\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.691345 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.691325    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkj5v\" (UniqueName: \"kubernetes.io/projected/a594de5b-9748-4355-86b8-95ca7dc9dda1-kube-api-access-bkj5v\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:56.691419 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.691334    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8xqw\" (UniqueName: \"kubernetes.io/projected/2758dbad-1f8a-4503-9dc2-6181518daeaf-kube-api-access-q8xqw\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:56.691587 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.691565    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2v9l\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-kube-api-access-k2v9l\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.691796 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.691776    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk26b\" (UniqueName: \"kubernetes.io/projected/e089d179-e583-4590-ba4c-39e7d486d707-kube-api-access-rk26b\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:56.691847 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.691831    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bm4p\" (UniqueName: \"kubernetes.io/projected/d31facfa-368e-4417-b9ce-30c3a495f650-kube-api-access-8bm4p\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.692140 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.692124    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzdfs\" (UniqueName: \"kubernetes.io/projected/79f0f110-a04f-4b35-8eba-c8f0469b62a1-kube-api-access-bzdfs\") pod \"kas-connection-checker-685895c7c9-q8mz6\" (UID: \"79f0f110-a04f-4b35-8eba-c8f0469b62a1\") " pod="kube-system/kas-connection-checker-685895c7c9-q8mz6"
Oct 30 12:35:56.693663 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.693648    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ddnv\" (UniqueName: \"kubernetes.io/projected/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-api-access-4ddnv\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:56.693721 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.693701    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h54zx\" (UniqueName: \"kubernetes.io/projected/eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d-kube-api-access-h54zx\") pod \"migrator-66df5d496b-tjlfm\" (UID: \"eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tjlfm"
Oct 30 12:35:56.693903 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.693886    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppcp2\" (UniqueName: \"kubernetes.io/projected/18fbde11-0050-4b55-bc23-a4e5e76ddc81-kube-api-access-ppcp2\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.694160 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.694145    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmn28\" (UniqueName: \"kubernetes.io/projected/e937e696-f801-42e7-9622-d774cca50028-kube-api-access-pmn28\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.694310 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.694295    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slh7l\" (UniqueName: \"kubernetes.io/projected/06910518-58db-450f-a5ee-521ac8f51e1a-kube-api-access-slh7l\") pod \"kas-connection-checker-685895c7c9-q74lv\" (UID: \"06910518-58db-450f-a5ee-521ac8f51e1a\") " pod="kube-system/kas-connection-checker-685895c7c9-q74lv"
Oct 30 12:35:56.694392 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.694367    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfh24\" (UniqueName: \"kubernetes.io/projected/4f13aca3-562e-406b-a154-8755b1bf7d15-kube-api-access-cfh24\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w\" (UID: \"4f13aca3-562e-406b-a154-8755b1bf7d15\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w"
Oct 30 12:35:56.694751 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.694737    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jknfr\" (UniqueName: \"kubernetes.io/projected/10a8a53c-d9d6-45aa-936e-48745236bb01-kube-api-access-jknfr\") pod \"network-check-source-6f78d59f99-gzkb8\" (UID: \"10a8a53c-d9d6-45aa-936e-48745236bb01\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-gzkb8"
Oct 30 12:35:56.774793 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.774773    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-cni-bin\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.774909 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.774798    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-modprobe-d\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.774909 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.774815    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-systemd-units\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.774909 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.774831    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.774909 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.774873    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-systemd-units\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.774909 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.774892    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-var-lib-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.774917    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.774893    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-cni-bin\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.774929    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-var-lib-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.774945    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-modprobe-d\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.774962    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-multus-certs\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.774932    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-multus-certs\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.774999    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775028    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775055    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-sys\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775078    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1263e83a-e0fd-4eb4-8141-5026aec6a631-host-slash\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775104    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775113    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-sys\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775121    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1263e83a-e0fd-4eb4-8141-5026aec6a631-host-slash\") pod \"iptables-alerter-k9987\" (UID: \"1263e83a-e0fd-4eb4-8141-5026aec6a631\") " pod="openshift-network-operator/iptables-alerter-k9987"
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775127    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.775128    4945 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:35:56.775164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775164    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-etc-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775190    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cnibin\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.775198    4945 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.775213    4945 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b7896669-ndwsr: secret "image-registry-tls" not found
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.775220    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert podName:49837f84-8643-4643-aa92-3716a9b755d1 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:57.275204168 +0000 UTC m=+1.853153256 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-44v66" (UID: "49837f84-8643-4643-aa92-3716a9b755d1") : secret "networking-console-plugin-cert" not found
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775231    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-cnibin\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.775130    4945 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775240    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-systemd\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775244    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-etc-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.775253    4945 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f558d755b-bj4vz: secret "image-registry-tls" not found
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.775274    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls podName:cc28fe61-b116-4a8b-ad04-123d62365a02 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:57.275258912 +0000 UTC m=+1.853208000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls") pod "image-registry-57b7896669-ndwsr" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02") : secret "image-registry-tls" not found
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775287    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-systemd\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775295    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.775335    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls podName:bbb14793-e4b1-4912-9ce2-008cba7ce1d4 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:57.275309088 +0000 UTC m=+1.853258168 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls") pod "image-registry-6f558d755b-bj4vz" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4") : secret "image-registry-tls" not found
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775342    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775357    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-etc-kubernetes\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775382    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775399    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-socket-dir-parent\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775425    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-kubelet\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775428    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-etc-kubernetes\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775451    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-cni-bin\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775467    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-kubelet\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775476    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-kubernetes\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.775507    4945 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775522    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-kubernetes\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775524    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-kubelet\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775542    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-cni-bin\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775553    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-log-socket\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775553    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-kubelet\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775543    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-socket-dir-parent\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.775563    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs podName:2758dbad-1f8a-4503-9dc2-6181518daeaf nodeName:}" failed. No retries permitted until 2026-10-30 12:35:57.275547471 +0000 UTC m=+1.853496558 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs") pod "network-metrics-daemon-xdcfv" (UID: "2758dbad-1f8a-4503-9dc2-6181518daeaf") : secret "metrics-daemon-secret" not found
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775622    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-log-socket\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775630    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-cnibin\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775656    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysctl-conf\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775680    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-cnibin\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775682    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775720    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/15bb3274-f845-4851-90ae-f4ab08e685c8-dbus\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.775745    4945 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:35:56.775746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775753    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b4171e4a-4d6e-4901-983a-600ab76ada45-crio-socket\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.775782    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls podName:e089d179-e583-4590-ba4c-39e7d486d707 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:57.275770456 +0000 UTC m=+1.853719538 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls") pod "dns-default-2djq9" (UID: "e089d179-e583-4590-ba4c-39e7d486d707") : secret "dns-default-metrics-tls" not found
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775799    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-run-netns\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775804    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b4171e4a-4d6e-4901-983a-600ab76ada45-crio-socket\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775805    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/15bb3274-f845-4851-90ae-f4ab08e685c8-dbus\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775822    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysctl-conf\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775825    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-cni-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775847    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-run-netns\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775854    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775876    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-registration-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775880    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-cni-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775893    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-cni-multus\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775898    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b4171e4a-4d6e-4901-983a-600ab76ada45-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775915    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-system-cni-dir\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775937    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b56b071-d626-4d8d-bba6-f5f69fa95db0-system-cni-dir\") pod \"multus-additional-cni-plugins-cpq89\" (UID: \"4b56b071-d626-4d8d-bba6-f5f69fa95db0\") " pod="openshift-multus/multus-additional-cni-plugins-cpq89"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775937    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-registration-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775950    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-k8s-cni-cncf-io\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775954    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-var-lib-cni-multus\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.775963    4945 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775973    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.775989    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-sys-fs\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776040    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-k8s-cni-cncf-io\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.776048    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls podName:3a2f595c-6efe-4b69-81bf-074985a6ddde nodeName:}" failed. No retries permitted until 2026-10-30 12:35:57.27600995 +0000 UTC m=+1.853959025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-48t4k" (UID: "3a2f595c-6efe-4b69-81bf-074985a6ddde") : secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776049    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-sys-fs\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776072    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-node-log\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776084    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-node-log\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776097    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776116    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776134    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-system-cni-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.776166    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:57.276156153 +0000 UTC m=+1.854105223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : configmap references non-existent config key: service-ca.crt
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776178    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-conf-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.776186    4945 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776195    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-device-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776212    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-netns\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776185    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-openvswitch\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776229    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-system-cni-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.776236    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls podName:b4171e4a-4d6e-4901-983a-600ab76ada45 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:57.276222208 +0000 UTC m=+1.854171286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lw4gw" (UID: "b4171e4a-4d6e-4901-983a-600ab76ada45") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776247    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-host-run-netns\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776251    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-multus-conf-dir\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776259    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776279    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776271    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-device-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776320    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-run-ovn-kubernetes\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776344    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-run-ovn-kubernetes\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776349    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.776415    4945 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.776441    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls podName:2be5910f-12fa-46fe-bf9a-ddff510f4d09 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:57.276433928 +0000 UTC m=+1.854383008 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-bdv44" (UID: "2be5910f-12fa-46fe-bf9a-ddff510f4d09") : secret "samples-operator-tls" not found
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776455    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysconfig\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776474    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-run\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776499    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-ovn\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776518    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-var-lib-kubelet\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776535    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-socket-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776550    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysconfig\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776555    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-etc-selinux\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776563    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-run-ovn\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776569    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-run\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776589    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-hosts-file\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776621    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-etc-selinux\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776634    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e937e696-f801-42e7-9622-d774cca50028-socket-dir\") pod \"aws-ebs-csi-driver-node-pppnh\" (UID: \"e937e696-f801-42e7-9622-d774cca50028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pppnh"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776636    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776686    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-hostroot\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776621    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-var-lib-kubelet\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776692    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0-hosts-file\") pod \"node-resolver-cwcj7\" (UID: \"e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0\") " pod="openshift-dns/node-resolver-cwcj7"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.776708    4945 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776739    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3ab05e61-6a22-4ced-abec-a528a15f9ba0-hostroot\") pod \"multus-kqrtg\" (UID: \"3ab05e61-6a22-4ced-abec-a528a15f9ba0\") " pod="openshift-multus/multus-kqrtg"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776745    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-systemd\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:56.776759    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:57.276746362 +0000 UTC m=+1.854695443 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : secret "router-metrics-certs-default" not found
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776709    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-systemd\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776787    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-lib-modules\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776805    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysctl-d\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776836    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-cni-netd\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776851    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-host\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776870    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a594de5b-9748-4355-86b8-95ca7dc9dda1-host\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776890    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-slash\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776899    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-lib-modules\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776912    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-cni-netd\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776915    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/15bb3274-f845-4851-90ae-f4ab08e685c8-kubelet-config\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776928    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-host\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776940    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a594de5b-9748-4355-86b8-95ca7dc9dda1-host\") pod \"node-ca-pt7kp\" (UID: \"a594de5b-9748-4355-86b8-95ca7dc9dda1\") " pod="openshift-image-registry/node-ca-pt7kp"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776950    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d31facfa-368e-4417-b9ce-30c3a495f650-etc-sysctl-d\") pod \"tuned-5frn2\" (UID: \"d31facfa-368e-4417-b9ce-30c3a495f650\") " pod="openshift-cluster-node-tuning-operator/tuned-5frn2"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776950    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/18fbde11-0050-4b55-bc23-a4e5e76ddc81-host-slash\") pod \"ovnkube-node-jk5qv\" (UID: \"18fbde11-0050-4b55-bc23-a4e5e76ddc81\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:56.777071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.776958    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/15bb3274-f845-4851-90ae-f4ab08e685c8-kubelet-config\") pod \"global-pull-secret-syncer-ntzbv\" (UID: \"15bb3274-f845-4851-90ae-f4ab08e685c8\") " pod="kube-system/global-pull-secret-syncer-ntzbv"
Oct 30 12:35:56.869875 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:56.869811    4945 scope.go:122] "RemoveContainer" containerID="962e40dd4a1466cf55a7d8df3804586fbec551f4fb8919a94013f65991c6296b"
Oct 30 12:35:57.053697 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.053670    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:57.280256 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.280222    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:57.280256 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.280258    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:57.280473 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.280280    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:57.280473 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.280301    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:57.280473 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.280327    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:57.280473 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.280354    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:57.280473 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.280367    4945 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:57.280473 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.280385    4945 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f558d755b-bj4vz: secret "image-registry-tls" not found
Oct 30 12:35:57.280473 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.280424    4945 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:35:57.280473 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.280430    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls podName:bbb14793-e4b1-4912-9ce2-008cba7ce1d4 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:58.280416352 +0000 UTC m=+2.858365433 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls") pod "image-registry-6f558d755b-bj4vz" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4") : secret "image-registry-tls" not found
Oct 30 12:35:57.280763 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.280487    4945 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:35:57.280763 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.280492    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:57.280763 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.280427    4945 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 12:35:57.280763 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.280428    4945 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:57.280763 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.280531    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert podName:49837f84-8643-4643-aa92-3716a9b755d1 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:58.280516318 +0000 UTC m=+2.858465402 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-44v66" (UID: "49837f84-8643-4643-aa92-3716a9b755d1") : secret "networking-console-plugin-cert" not found
Oct 30 12:35:57.280763 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.280548    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:57.280763 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.280556    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls podName:3a2f595c-6efe-4b69-81bf-074985a6ddde nodeName:}" failed. No retries permitted until 2026-10-30 12:35:58.280546544 +0000 UTC m=+2.858495625 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-48t4k" (UID: "3a2f595c-6efe-4b69-81bf-074985a6ddde") : secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:57.280763 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.280369    4945 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:57.280763 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.280572    4945 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b7896669-ndwsr: secret "image-registry-tls" not found
Oct 30 12:35:57.280763 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.280615    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls podName:e089d179-e583-4590-ba4c-39e7d486d707 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:58.280584422 +0000 UTC m=+2.858533512 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls") pod "dns-default-2djq9" (UID: "e089d179-e583-4590-ba4c-39e7d486d707") : secret "dns-default-metrics-tls" not found
Oct 30 12:35:57.280763 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.280621    4945 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:35:57.280763 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.280640    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs podName:2758dbad-1f8a-4503-9dc2-6181518daeaf nodeName:}" failed. No retries permitted until 2026-10-30 12:35:58.280634431 +0000 UTC m=+2.858583498 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs") pod "network-metrics-daemon-xdcfv" (UID: "2758dbad-1f8a-4503-9dc2-6181518daeaf") : secret "metrics-daemon-secret" not found
Oct 30 12:35:57.281158 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.280652    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:58.280647374 +0000 UTC m=+2.858596441 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : configmap references non-existent config key: service-ca.crt
Oct 30 12:35:57.281158 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.281069    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls podName:cc28fe61-b116-4a8b-ad04-123d62365a02 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:58.281062555 +0000 UTC m=+2.859011622 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls") pod "image-registry-57b7896669-ndwsr" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02") : secret "image-registry-tls" not found
Oct 30 12:35:57.281158 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.281087    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls podName:b4171e4a-4d6e-4901-983a-600ab76ada45 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:58.281081576 +0000 UTC m=+2.859030643 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lw4gw" (UID: "b4171e4a-4d6e-4901-983a-600ab76ada45") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:35:57.281158 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.281124    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:57.281158 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.281153    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:57.281315 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.281238    4945 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:35:57.281315 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.281240    4945 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:35:57.281315 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.281262    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:58.281255963 +0000 UTC m=+2.859205030 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : secret "router-metrics-certs-default" not found
Oct 30 12:35:57.281315 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.281284    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls podName:2be5910f-12fa-46fe-bf9a-ddff510f4d09 nodeName:}" failed. No retries permitted until 2026-10-30 12:35:58.281273541 +0000 UTC m=+2.859222608 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-bdv44" (UID: "2be5910f-12fa-46fe-bf9a-ddff510f4d09") : secret "samples-operator-tls" not found
Oct 30 12:35:57.604667 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.604560    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 12:35:57.605714 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.605689    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/1.log"
Oct 30 12:35:57.605844 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.605725    4945 generic.go:356] "Generic (PLEG): container finished" podID="b80ffbfe-445f-431c-b8f4-be9fd2c5d094" containerID="4ba38da302f5a03a4e704eb55860ed1348b9a842a93a29062dfcd10f64adfeb8" exitCode=255
Oct 30 12:35:57.605844 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.605823    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8" event={"ID":"b80ffbfe-445f-431c-b8f4-be9fd2c5d094","Type":"ContainerDied","Data":"4ba38da302f5a03a4e704eb55860ed1348b9a842a93a29062dfcd10f64adfeb8"}
Oct 30 12:35:57.605950 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.605913    4945 scope.go:122] "RemoveContainer" containerID="962e40dd4a1466cf55a7d8df3804586fbec551f4fb8919a94013f65991c6296b"
Oct 30 12:35:57.606487 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.606467    4945 scope.go:122] "RemoveContainer" containerID="4ba38da302f5a03a4e704eb55860ed1348b9a842a93a29062dfcd10f64adfeb8"
Oct 30 12:35:57.606678 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:57.606658    4945 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-8gvm8_openshift-console-operator(b80ffbfe-445f-431c-b8f4-be9fd2c5d094)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8" podUID="b80ffbfe-445f-431c-b8f4-be9fd2c5d094"
Oct 30 12:35:57.963504 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:57.963477    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cwcj7_e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0/dns-node-resolver/0.log"
Oct 30 12:35:58.291693 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:58.291585    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:35:58.291693 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:58.291640    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:35:58.291693 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:58.291667    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:35:58.291693 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:58.291687    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:58.291711    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:58.291731    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291752    4945 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291804    4945 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291813    4945 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:58.291759    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291827    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert podName:49837f84-8643-4643-aa92-3716a9b755d1 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.291805807 +0000 UTC m=+4.869754894 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-44v66" (UID: "49837f84-8643-4643-aa92-3716a9b755d1") : secret "networking-console-plugin-cert" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291833    4945 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b7896669-ndwsr: secret "image-registry-tls" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291822    4945 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f558d755b-bj4vz: secret "image-registry-tls" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291853    4945 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291833    4945 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:58.291868    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291833    4945 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291892    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls podName:cc28fe61-b116-4a8b-ad04-123d62365a02 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.291874876 +0000 UTC m=+4.869823945 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls") pod "image-registry-57b7896669-ndwsr" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02") : secret "image-registry-tls" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291920    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls podName:e089d179-e583-4590-ba4c-39e7d486d707 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.291913051 +0000 UTC m=+4.869862119 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls") pod "dns-default-2djq9" (UID: "e089d179-e583-4590-ba4c-39e7d486d707") : secret "dns-default-metrics-tls" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291923    4945 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291934    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.291928409 +0000 UTC m=+4.869877476 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : configmap references non-existent config key: service-ca.crt
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291946    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs podName:2758dbad-1f8a-4503-9dc2-6181518daeaf nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.291941007 +0000 UTC m=+4.869890074 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs") pod "network-metrics-daemon-xdcfv" (UID: "2758dbad-1f8a-4503-9dc2-6181518daeaf") : secret "metrics-daemon-secret" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291962    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls podName:3a2f595c-6efe-4b69-81bf-074985a6ddde nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.291952604 +0000 UTC m=+4.869901671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-48t4k" (UID: "3a2f595c-6efe-4b69-81bf-074985a6ddde") : secret "cluster-monitoring-operator-tls" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.291979    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls podName:bbb14793-e4b1-4912-9ce2-008cba7ce1d4 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.291971023 +0000 UTC m=+4.869920096 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls") pod "image-registry-6f558d755b-bj4vz" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4") : secret "image-registry-tls" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.292004    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls podName:b4171e4a-4d6e-4901-983a-600ab76ada45 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.291995137 +0000 UTC m=+4.869944206 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lw4gw" (UID: "b4171e4a-4d6e-4901-983a-600ab76ada45") : secret "insights-runtime-extractor-tls" not found
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:58.292050    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:35:58.292090 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:58.292081    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:35:58.292837 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.292172    4945 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 12:35:58.292837 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.292178    4945 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:35:58.292837 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.292217    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls podName:2be5910f-12fa-46fe-bf9a-ddff510f4d09 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.292205441 +0000 UTC m=+4.870154522 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-bdv44" (UID: "2be5910f-12fa-46fe-bf9a-ddff510f4d09") : secret "samples-operator-tls" not found
Oct 30 12:35:58.292837 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:35:58.292233    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:00.292227175 +0000 UTC m=+4.870176255 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : secret "router-metrics-certs-default" not found
Oct 30 12:35:58.610188 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:58.610120    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 12:35:58.963832 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:58.963810    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pt7kp_a594de5b-9748-4355-86b8-95ca7dc9dda1/node-ca/0.log"
Oct 30 12:35:59.103783 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:59.103757    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:35:59.103974 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:59.103948    4945 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 12:35:59.563595 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:59.563546    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t5j98_10337f54-95e9-42cd-a73f-16817fc1cb96/serve-healthcheck-canary/0.log"
Oct 30 12:35:59.964932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:35:59.964893    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-tjlfm_eaeda6ae-9b69-4cd2-b57b-78dec64c9c7d/migrator/0.log"
Oct 30 12:36:00.167779 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.167749    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w_4f13aca3-562e-406b-a154-8755b1bf7d15/kube-storage-version-migrator-operator/0.log"
Oct 30 12:36:00.309265 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.309179    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:36:00.309265 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.309224    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:36:00.309265 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.309260    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:36:00.309557 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.309289    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:36:00.309557 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.309322    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:36:00.309557 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:00.309333    4945 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:36:00.309557 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.309353    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:36:00.309557 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:00.309395    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls podName:e089d179-e583-4590-ba4c-39e7d486d707 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:04.309381492 +0000 UTC m=+8.887330574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls") pod "dns-default-2djq9" (UID: "e089d179-e583-4590-ba4c-39e7d486d707") : secret "dns-default-metrics-tls" not found
Oct 30 12:36:00.309557 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:00.309432    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:04.30941525 +0000 UTC m=+8.887364338 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : configmap references non-existent config key: service-ca.crt
Oct 30 12:36:00.309557 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:00.309441    4945 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:36:00.309557 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:00.309486    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:04.309471838 +0000 UTC m=+8.887420918 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : secret "router-metrics-certs-default" not found
Oct 30 12:36:00.309557 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.309481    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:36:00.309557 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.309515    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:36:00.309557 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.309551    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:36:00.310136 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.309591    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:36:00.310136 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:00.309660    4945 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:36:00.310136 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:00.309675    4945 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f558d755b-bj4vz: secret "image-registry-tls" not found
Oct 30 12:36:00.310136 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:00.309689    4945 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:36:00.310136 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:00.309702    4945 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b7896669-ndwsr: secret "image-registry-tls" not found
Oct 30 12:36:00.310136 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:00.309708    4945 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 12:36:00.310136 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:00.309728    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls podName:bbb14793-e4b1-4912-9ce2-008cba7ce1d4 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:04.309712659 +0000 UTC m=+8.887661743 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls") pod "image-registry-6f558d755b-bj4vz" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4") : secret "image-registry-tls" not found
Oct 30 12:36:00.310136 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.309731    4945 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 12:36:00.310136 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:00.309747    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls podName:cc28fe61-b116-4a8b-ad04-123d62365a02 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:04.309741491 +0000 UTC m=+8.887690588 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls") pod "image-registry-57b7896669-ndwsr" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02") : secret "image-registry-tls" not found
Oct 30 12:36:00.310136 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:00.309758    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs podName:2758dbad-1f8a-4503-9dc2-6181518daeaf nodeName:}" failed. No retries permitted until 2026-10-30 12:36:04.309753066 +0000 UTC m=+8.887702133 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs") pod "network-metrics-daemon-xdcfv" (UID: "2758dbad-1f8a-4503-9dc2-6181518daeaf") : secret "metrics-daemon-secret" not found
Oct 30 12:36:00.311995 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.311975    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3a2f595c-6efe-4b69-81bf-074985a6ddde-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-48t4k\" (UID: \"3a2f595c-6efe-4b69-81bf-074985a6ddde\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:36:00.312094 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.311989    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/2be5910f-12fa-46fe-bf9a-ddff510f4d09-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-bdv44\" (UID: \"2be5910f-12fa-46fe-bf9a-ddff510f4d09\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:36:00.312158 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.312139    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b4171e4a-4d6e-4901-983a-600ab76ada45-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lw4gw\" (UID: \"b4171e4a-4d6e-4901-983a-600ab76ada45\") " pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:36:00.312241 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.312221    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/49837f84-8643-4643-aa92-3716a9b755d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-44v66\" (UID: \"49837f84-8643-4643-aa92-3716a9b755d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:36:00.465920 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.465896    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"
Oct 30 12:36:00.471349 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.471328    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"
Oct 30 12:36:00.472423 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.472406    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66"
Oct 30 12:36:00.480058 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.480037    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lw4gw"
Oct 30 12:36:00.765293 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.765264    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44"]
Oct 30 12:36:00.774940 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.774917    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k"]
Oct 30 12:36:00.788134 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.788111    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-44v66"]
Oct 30 12:36:00.789899 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:36:00.789865    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a2f595c_6efe_4b69_81bf_074985a6ddde.slice/crio-4679986f85980a1c6a6edf67a152c454717804948fd80011bdf1c8037de1f836 WatchSource:0}: Error finding container 4679986f85980a1c6a6edf67a152c454717804948fd80011bdf1c8037de1f836: Status 404 returned error can't find the container with id 4679986f85980a1c6a6edf67a152c454717804948fd80011bdf1c8037de1f836
Oct 30 12:36:00.791468 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.791444    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lw4gw"]
Oct 30 12:36:00.792652 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:00.792629    4945 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 12:36:00.792900 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:36:00.792876    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49837f84_8643_4643_aa92_3716a9b755d1.slice/crio-e7e26ac6fb182a4c14694b52787fb4b81029f11e88b0e3f876ad3f807c1977ba WatchSource:0}: Error finding container e7e26ac6fb182a4c14694b52787fb4b81029f11e88b0e3f876ad3f807c1977ba: Status 404 returned error can't find the container with id e7e26ac6fb182a4c14694b52787fb4b81029f11e88b0e3f876ad3f807c1977ba
Oct 30 12:36:00.795512 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:36:00.795490    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4171e4a_4d6e_4901_983a_600ab76ada45.slice/crio-f2d01cf64f483a9caa08f957ba2c7067560e1b069c565d4d59bba36b16928f5b WatchSource:0}: Error finding container f2d01cf64f483a9caa08f957ba2c7067560e1b069c565d4d59bba36b16928f5b: Status 404 returned error can't find the container with id f2d01cf64f483a9caa08f957ba2c7067560e1b069c565d4d59bba36b16928f5b
Oct 30 12:36:01.622060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:01.622028    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lw4gw" event={"ID":"b4171e4a-4d6e-4901-983a-600ab76ada45","Type":"ContainerStarted","Data":"a2a536c0f402ca75fb06f88c7e9359e30957eb6b04d0f2383eb4c831d0ffae76"}
Oct 30 12:36:01.622414 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:01.622113    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lw4gw" event={"ID":"b4171e4a-4d6e-4901-983a-600ab76ada45","Type":"ContainerStarted","Data":"f2d01cf64f483a9caa08f957ba2c7067560e1b069c565d4d59bba36b16928f5b"}
Oct 30 12:36:01.624499 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:01.624472    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66" event={"ID":"49837f84-8643-4643-aa92-3716a9b755d1","Type":"ContainerStarted","Data":"e7e26ac6fb182a4c14694b52787fb4b81029f11e88b0e3f876ad3f807c1977ba"}
Oct 30 12:36:01.626969 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:01.626907    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k" event={"ID":"3a2f595c-6efe-4b69-81bf-074985a6ddde","Type":"ContainerStarted","Data":"4679986f85980a1c6a6edf67a152c454717804948fd80011bdf1c8037de1f836"}
Oct 30 12:36:01.629113 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:01.629089    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44" event={"ID":"2be5910f-12fa-46fe-bf9a-ddff510f4d09","Type":"ContainerStarted","Data":"321f064d1428f70e37673aa4b846f711892fe95e70f337fa8914ff520a9a7140"}
Oct 30 12:36:02.572060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:02.571972    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:36:02.572459 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:02.572438    4945 scope.go:122] "RemoveContainer" containerID="4ba38da302f5a03a4e704eb55860ed1348b9a842a93a29062dfcd10f64adfeb8"
Oct 30 12:36:02.573046 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:02.572695    4945 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-8gvm8_openshift-console-operator(b80ffbfe-445f-431c-b8f4-be9fd2c5d094)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8" podUID="b80ffbfe-445f-431c-b8f4-be9fd2c5d094"
Oct 30 12:36:02.636867 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:02.636786    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lw4gw" event={"ID":"b4171e4a-4d6e-4901-983a-600ab76ada45","Type":"ContainerStarted","Data":"2f97e6ab306bf6f9354c414e1c32809254160bc69759ac85443ab70a9f9ac143"}
Oct 30 12:36:02.639382 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:02.639348    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-44v66" event={"ID":"49837f84-8643-4643-aa92-3716a9b755d1","Type":"ContainerStarted","Data":"50016a2af38d40c47e7d1b9e8e18b8837bd6c2a853a034270bf509ee3e03104a"}
Oct 30 12:36:03.645476 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:03.645442    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lw4gw" event={"ID":"b4171e4a-4d6e-4901-983a-600ab76ada45","Type":"ContainerStarted","Data":"5cc0958168e00eee9ef4faabe6edf845a26af1b57fb74fca97d94221080dad7a"}
Oct 30 12:36:03.647233 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:03.647206    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-48t4k" event={"ID":"3a2f595c-6efe-4b69-81bf-074985a6ddde","Type":"ContainerStarted","Data":"64d80c580aaa26f499c1a7af86b2db8c6e59ec60dbae89b940900ab8bf7087b1"}
Oct 30 12:36:03.649008 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:03.648986    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44" event={"ID":"2be5910f-12fa-46fe-bf9a-ddff510f4d09","Type":"ContainerStarted","Data":"acd72e5f9945f4ea5b208372ea60c100b9790f0e9f3978f53e565b5ed4df4360"}
Oct 30 12:36:03.649087 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:03.649030    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-bdv44" event={"ID":"2be5910f-12fa-46fe-bf9a-ddff510f4d09","Type":"ContainerStarted","Data":"971f62f7d828b9d9948879a120c43b2fc450abfb67f1f96458fa773774d31c33"}
Oct 30 12:36:04.260522 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:04.260488    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:36:04.325910 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:04.325877    4945 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:36:04.326218 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:04.326205    4945 scope.go:122] "RemoveContainer" containerID="4ba38da302f5a03a4e704eb55860ed1348b9a842a93a29062dfcd10f64adfeb8"
Oct 30 12:36:04.326378 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:04.326361    4945 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-8gvm8_openshift-console-operator(b80ffbfe-445f-431c-b8f4-be9fd2c5d094)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8" podUID="b80ffbfe-445f-431c-b8f4-be9fd2c5d094"
Oct 30 12:36:04.349876 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:04.349848    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:36:04.350012 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:04.349884    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:36:04.350012 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:04.349908    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:36:04.350012 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:04.349936    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:36:04.350012 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:04.349966    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:36:04.350012 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:04.349989    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:36:04.350260 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:04.350020    4945 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 12:36:04.350260 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:04.350067    4945 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:36:04.350260 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:04.350079    4945 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57b7896669-ndwsr: secret "image-registry-tls" not found
Oct 30 12:36:04.350260 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:04.350071    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:12.350048981 +0000 UTC m=+16.927998066 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : configmap references non-existent config key: service-ca.crt
Oct 30 12:36:04.350260 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:04.350119    4945 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 12:36:04.350260 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:04.350131    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs podName:d81730d3-53ed-40b8-a512-348fc042f821 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:12.350118107 +0000 UTC m=+16.928067189 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs") pod "router-default-846cf5b65-gllvj" (UID: "d81730d3-53ed-40b8-a512-348fc042f821") : secret "router-metrics-certs-default" not found
Oct 30 12:36:04.350260 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:04.350144    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls podName:cc28fe61-b116-4a8b-ad04-123d62365a02 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:12.350137784 +0000 UTC m=+16.928086886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls") pod "image-registry-57b7896669-ndwsr" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02") : secret "image-registry-tls" not found
Oct 30 12:36:04.350260 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:04.350145    4945 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 12:36:04.350260 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:04.350163    4945 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f558d755b-bj4vz: secret "image-registry-tls" not found
Oct 30 12:36:04.350260 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:04.350168    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls podName:e089d179-e583-4590-ba4c-39e7d486d707 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:12.350156868 +0000 UTC m=+16.928105950 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls") pod "dns-default-2djq9" (UID: "e089d179-e583-4590-ba4c-39e7d486d707") : secret "dns-default-metrics-tls" not found
Oct 30 12:36:04.350260 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:04.350216    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls podName:bbb14793-e4b1-4912-9ce2-008cba7ce1d4 nodeName:}" failed. No retries permitted until 2026-10-30 12:36:12.350204736 +0000 UTC m=+16.928153822 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls") pod "image-registry-6f558d755b-bj4vz" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4") : secret "image-registry-tls" not found
Oct 30 12:36:04.352331 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:04.352299    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2758dbad-1f8a-4503-9dc2-6181518daeaf-metrics-certs\") pod \"network-metrics-daemon-xdcfv\" (UID: \"2758dbad-1f8a-4503-9dc2-6181518daeaf\") " pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:36:04.378347 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:04.378324    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdcfv"
Oct 30 12:36:04.666579 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:04.666502    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xdcfv"]
Oct 30 12:36:04.670827 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:36:04.670801    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2758dbad_1f8a_4503_9dc2_6181518daeaf.slice/crio-eb24da7851d038768fe0800923d28decb462da7994e05a7edab7b75b59949a42 WatchSource:0}: Error finding container eb24da7851d038768fe0800923d28decb462da7994e05a7edab7b75b59949a42: Status 404 returned error can't find the container with id eb24da7851d038768fe0800923d28decb462da7994e05a7edab7b75b59949a42
Oct 30 12:36:05.657891 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:05.657855    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xdcfv" event={"ID":"2758dbad-1f8a-4503-9dc2-6181518daeaf","Type":"ContainerStarted","Data":"aa4b854f54187ce16b8efbb4d2e6942e8830309accbb47f0d609d5ba7f60637e"}
Oct 30 12:36:05.657990 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:05.657923    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xdcfv" event={"ID":"2758dbad-1f8a-4503-9dc2-6181518daeaf","Type":"ContainerStarted","Data":"eb24da7851d038768fe0800923d28decb462da7994e05a7edab7b75b59949a42"}
Oct 30 12:36:06.166511 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:06.166485    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jk5qv"
Oct 30 12:36:06.663645 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:06.663590    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xdcfv" event={"ID":"2758dbad-1f8a-4503-9dc2-6181518daeaf","Type":"ContainerStarted","Data":"550e9453580f063a64e41b80d81f7653f1dffcae5ad9d514419f34c2dd9b036b"}
Oct 30 12:36:07.541767 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:07.541731    4945 scope.go:122] "RemoveContainer" containerID="4ba38da302f5a03a4e704eb55860ed1348b9a842a93a29062dfcd10f64adfeb8"
Oct 30 12:36:07.668492 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:07.668466    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 12:36:07.668653 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:07.668544    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8" event={"ID":"b80ffbfe-445f-431c-b8f4-be9fd2c5d094","Type":"ContainerStarted","Data":"6dc5c15609a7e97f41ca3a4b9aceb5e8f799fa7b85176bfdfbc4140224619973"}
Oct 30 12:36:12.417026 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.416992    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:36:12.417505 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.417042    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:36:12.417505 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.417068    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:36:12.417505 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.417112    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:36:12.417505 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.417149    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:36:12.417891 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.417867    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d81730d3-53ed-40b8-a512-348fc042f821-service-ca-bundle\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:36:12.419551 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.419523    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d81730d3-53ed-40b8-a512-348fc042f821-metrics-certs\") pod \"router-default-846cf5b65-gllvj\" (UID: \"d81730d3-53ed-40b8-a512-348fc042f821\") " pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:36:12.419654 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.419523    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") pod \"image-registry-57b7896669-ndwsr\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") " pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:36:12.419794 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.419771    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e089d179-e583-4590-ba4c-39e7d486d707-metrics-tls\") pod \"dns-default-2djq9\" (UID: \"e089d179-e583-4590-ba4c-39e7d486d707\") " pod="openshift-dns/dns-default-2djq9"
Oct 30 12:36:12.420257 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.420237    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") pod \"image-registry-6f558d755b-bj4vz\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") " pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:36:12.470409 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.470389    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:36:12.471476 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.471454    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:36:12.475395 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.475380    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:36:12.479560 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.479538    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2djq9"
Oct 30 12:36:12.809330 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.809281    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-846cf5b65-gllvj"]
Oct 30 12:36:12.813241 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:36:12.813214    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd81730d3_53ed_40b8_a512_348fc042f821.slice/crio-9c322c4d2507611f8961f0227a3363ce8f38faa5379c432607f3d413c12a106f WatchSource:0}: Error finding container 9c322c4d2507611f8961f0227a3363ce8f38faa5379c432607f3d413c12a106f: Status 404 returned error can't find the container with id 9c322c4d2507611f8961f0227a3363ce8f38faa5379c432607f3d413c12a106f
Oct 30 12:36:12.821429 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.821401    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57b7896669-ndwsr"]
Oct 30 12:36:12.822057 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.822035    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2djq9"]
Oct 30 12:36:12.825558 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:36:12.825534    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc28fe61_b116_4a8b_ad04_123d62365a02.slice/crio-913d62efb8e205efcfa16791e0fe86f76b779ed9a9444ad68319b6b7f3b4be29 WatchSource:0}: Error finding container 913d62efb8e205efcfa16791e0fe86f76b779ed9a9444ad68319b6b7f3b4be29: Status 404 returned error can't find the container with id 913d62efb8e205efcfa16791e0fe86f76b779ed9a9444ad68319b6b7f3b4be29
Oct 30 12:36:12.827316 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:36:12.827295    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode089d179_e583_4590_ba4c_39e7d486d707.slice/crio-2f4a8c108b53abf36e65e638e00a7d9f2fa9b277a867f48399e0d7797014ee5c WatchSource:0}: Error finding container 2f4a8c108b53abf36e65e638e00a7d9f2fa9b277a867f48399e0d7797014ee5c: Status 404 returned error can't find the container with id 2f4a8c108b53abf36e65e638e00a7d9f2fa9b277a867f48399e0d7797014ee5c
Oct 30 12:36:12.829682 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:36:12.829661    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbb14793_e4b1_4912_9ce2_008cba7ce1d4.slice/crio-31268150c1b1a09fda8387ec5da6dbd041507fdc60666d9d037fbafa577733e1 WatchSource:0}: Error finding container 31268150c1b1a09fda8387ec5da6dbd041507fdc60666d9d037fbafa577733e1: Status 404 returned error can't find the container with id 31268150c1b1a09fda8387ec5da6dbd041507fdc60666d9d037fbafa577733e1
Oct 30 12:36:12.832873 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:12.832853    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f558d755b-bj4vz"]
Oct 30 12:36:13.692376 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:13.692330    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57b7896669-ndwsr" event={"ID":"cc28fe61-b116-4a8b-ad04-123d62365a02","Type":"ContainerStarted","Data":"6cc54aec1b1104b8c2d3407f353d2eac0a4bb74889e540761b152a9292075dde"}
Oct 30 12:36:13.692943 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:13.692397    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57b7896669-ndwsr" event={"ID":"cc28fe61-b116-4a8b-ad04-123d62365a02","Type":"ContainerStarted","Data":"913d62efb8e205efcfa16791e0fe86f76b779ed9a9444ad68319b6b7f3b4be29"}
Oct 30 12:36:13.692943 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:13.692430    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:36:13.694332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:13.694300    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-846cf5b65-gllvj" event={"ID":"d81730d3-53ed-40b8-a512-348fc042f821","Type":"ContainerStarted","Data":"1896e22225aa2085d9b8bd33656c76331cbf829ae6be5ad1499fc49e0814622a"}
Oct 30 12:36:13.694463 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:13.694349    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-846cf5b65-gllvj" event={"ID":"d81730d3-53ed-40b8-a512-348fc042f821","Type":"ContainerStarted","Data":"9c322c4d2507611f8961f0227a3363ce8f38faa5379c432607f3d413c12a106f"}
Oct 30 12:36:13.695510 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:13.695483    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2djq9" event={"ID":"e089d179-e583-4590-ba4c-39e7d486d707","Type":"ContainerStarted","Data":"2f4a8c108b53abf36e65e638e00a7d9f2fa9b277a867f48399e0d7797014ee5c"}
Oct 30 12:36:13.697926 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:13.697406    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f558d755b-bj4vz" event={"ID":"bbb14793-e4b1-4912-9ce2-008cba7ce1d4","Type":"ContainerStarted","Data":"3a7bc4e2c2d03c790ca77f4b3b80ffa22b797be90efca42f365c1a0c65efd1dd"}
Oct 30 12:36:13.697926 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:13.697447    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f558d755b-bj4vz" event={"ID":"bbb14793-e4b1-4912-9ce2-008cba7ce1d4","Type":"ContainerStarted","Data":"31268150c1b1a09fda8387ec5da6dbd041507fdc60666d9d037fbafa577733e1"}
Oct 30 12:36:13.697926 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:13.697880    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:36:14.471096 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:14.471064    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:36:14.473741 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:14.473720    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:36:14.703537 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:14.703492    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2djq9" event={"ID":"e089d179-e583-4590-ba4c-39e7d486d707","Type":"ContainerStarted","Data":"bfb554a509f2c0ab1ea0782ede0a92ed16b1092267ddf3e913a763b09199c00e"}
Oct 30 12:36:14.703988 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:14.703562    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2djq9" event={"ID":"e089d179-e583-4590-ba4c-39e7d486d707","Type":"ContainerStarted","Data":"fb2adb3514770834b7637c4d57540779a1d36606d18492e7cfd5f82d3f926880"}
Oct 30 12:36:14.703988 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:14.703801    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2djq9"
Oct 30 12:36:14.703988 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:14.703926    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:36:14.705090 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:14.705076    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-846cf5b65-gllvj"
Oct 30 12:36:15.306622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:15.306563    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f558d755b-bj4vz"]
Oct 30 12:36:17.669175 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:17.669140    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:36:17.672900 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:17.672883    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-8gvm8"
Oct 30 12:36:20.465899 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:20.465869    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p2z49"
Oct 30 12:36:23.654392 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.654352    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-p9pcf"]
Oct 30 12:36:23.707545 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.707516    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.733593 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.733568    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 12:36:23.733739 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.733685    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-87th4\""
Oct 30 12:36:23.733739 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.733725    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 12:36:23.733849 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.733784    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 12:36:23.745413 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.745384    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 12:36:23.801764 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.801733    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6a83a03a-d39a-4814-bfa0-735fdd994de6-node-exporter-wtmp\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.801878 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.801777    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a83a03a-d39a-4814-bfa0-735fdd994de6-sys\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.801878 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.801844    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6a83a03a-d39a-4814-bfa0-735fdd994de6-node-exporter-tls\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.801878 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.801870    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6a83a03a-d39a-4814-bfa0-735fdd994de6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.802037 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.801902    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6a83a03a-d39a-4814-bfa0-735fdd994de6-root\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.802037 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.801968    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6a83a03a-d39a-4814-bfa0-735fdd994de6-node-exporter-accelerators-collector-config\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.802141 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.802048    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cftwb\" (UniqueName: \"kubernetes.io/projected/6a83a03a-d39a-4814-bfa0-735fdd994de6-kube-api-access-cftwb\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.802141 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.802087    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6a83a03a-d39a-4814-bfa0-735fdd994de6-node-exporter-textfile\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.802141 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.802129    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a83a03a-d39a-4814-bfa0-735fdd994de6-metrics-client-ca\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.903091 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.903058    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6a83a03a-d39a-4814-bfa0-735fdd994de6-node-exporter-wtmp\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.903218 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.903117    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a83a03a-d39a-4814-bfa0-735fdd994de6-sys\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.903218 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.903153    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6a83a03a-d39a-4814-bfa0-735fdd994de6-node-exporter-tls\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.903218 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.903183    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6a83a03a-d39a-4814-bfa0-735fdd994de6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.903378 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.903219    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6a83a03a-d39a-4814-bfa0-735fdd994de6-root\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.903378 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.903230    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6a83a03a-d39a-4814-bfa0-735fdd994de6-node-exporter-wtmp\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.903378 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.903251    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6a83a03a-d39a-4814-bfa0-735fdd994de6-node-exporter-accelerators-collector-config\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.903378 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.903230    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a83a03a-d39a-4814-bfa0-735fdd994de6-sys\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.903378 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.903284    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cftwb\" (UniqueName: \"kubernetes.io/projected/6a83a03a-d39a-4814-bfa0-735fdd994de6-kube-api-access-cftwb\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.903378 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.903316    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6a83a03a-d39a-4814-bfa0-735fdd994de6-root\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.903378 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.903321    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6a83a03a-d39a-4814-bfa0-735fdd994de6-node-exporter-textfile\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.903726 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.903432    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a83a03a-d39a-4814-bfa0-735fdd994de6-metrics-client-ca\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.903814 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.903788    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6a83a03a-d39a-4814-bfa0-735fdd994de6-node-exporter-textfile\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.904014 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.903995    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6a83a03a-d39a-4814-bfa0-735fdd994de6-node-exporter-accelerators-collector-config\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.904088 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.904066    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a83a03a-d39a-4814-bfa0-735fdd994de6-metrics-client-ca\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.905585 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.905533    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6a83a03a-d39a-4814-bfa0-735fdd994de6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.905770 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.905751    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6a83a03a-d39a-4814-bfa0-735fdd994de6-node-exporter-tls\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:23.921401 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:23.921381    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cftwb\" (UniqueName: \"kubernetes.io/projected/6a83a03a-d39a-4814-bfa0-735fdd994de6-kube-api-access-cftwb\") pod \"node-exporter-p9pcf\" (UID: \"6a83a03a-d39a-4814-bfa0-735fdd994de6\") " pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:24.058576 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:24.058554    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p9pcf"
Oct 30 12:36:24.088791 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:36:24.088759    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a83a03a_d39a_4814_bfa0_735fdd994de6.slice/crio-1d770e9521f321fce86f40c53ceb086becced1a66fe3ab30a9d958172869502f WatchSource:0}: Error finding container 1d770e9521f321fce86f40c53ceb086becced1a66fe3ab30a9d958172869502f: Status 404 returned error can't find the container with id 1d770e9521f321fce86f40c53ceb086becced1a66fe3ab30a9d958172869502f
Oct 30 12:36:24.263175 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:24.263141    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-29mpd"
Oct 30 12:36:24.709415 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:24.709382    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2djq9"
Oct 30 12:36:24.741782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:24.741752    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p9pcf" event={"ID":"6a83a03a-d39a-4814-bfa0-735fdd994de6","Type":"ContainerStarted","Data":"1d770e9521f321fce86f40c53ceb086becced1a66fe3ab30a9d958172869502f"}
Oct 30 12:36:25.748933 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:25.748903    4945 generic.go:356] "Generic (PLEG): container finished" podID="6a83a03a-d39a-4814-bfa0-735fdd994de6" containerID="7c1919e6d2197bc9287da4cc869a2d74dfda75e486778e2b59a68cf8362f61d3" exitCode=0
Oct 30 12:36:25.749364 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:25.748954    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p9pcf" event={"ID":"6a83a03a-d39a-4814-bfa0-735fdd994de6","Type":"ContainerDied","Data":"7c1919e6d2197bc9287da4cc869a2d74dfda75e486778e2b59a68cf8362f61d3"}
Oct 30 12:36:26.190836 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:26.190762    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:36:26.754700 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:26.754660    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p9pcf" event={"ID":"6a83a03a-d39a-4814-bfa0-735fdd994de6","Type":"ContainerStarted","Data":"eceb70fbe5cec1aac023b9ef12ccc2ab7f2aadfe9fdbcb18c1cefb21d3cc2b83"}
Oct 30 12:36:26.755192 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:26.754737    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p9pcf" event={"ID":"6a83a03a-d39a-4814-bfa0-735fdd994de6","Type":"ContainerStarted","Data":"2734703d7e2b60e63a58bf2e5335b1d46573b236b7902aa7d1ee9e5f6eed43c4"}
Oct 30 12:36:26.795821 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:26.795768    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-p9pcf" podStartSLOduration=2.386870095 podStartE2EDuration="3.795750999s" totalImagesPullingTime="1.322687472s" totalInitContainerRuntime="86.193432ms" isStatefulPod=true podCreationTimestamp="2026-10-30 12:36:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:36:26.794765676 +0000 UTC m=+31.372714766" watchObservedRunningTime="2026-10-30 12:36:26.795750999 +0000 UTC m=+31.373700067"
Oct 30 12:36:32.480300 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:32.480261    4945 patch_prober.go:29] interesting pod/image-registry-57b7896669-ndwsr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 12:36:32.480681 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:32.480330    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-57b7896669-ndwsr" podUID="cc28fe61-b116-4a8b-ad04-123d62365a02" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 12:36:34.708639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:34.708592    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:36:36.715669 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:36.715579    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:36:38.550381 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:38.550342    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57b7896669-ndwsr"]
Oct 30 12:36:41.729550 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:41.729500    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6f558d755b-bj4vz" podUID="bbb14793-e4b1-4912-9ce2-008cba7ce1d4" containerName="registry" containerID="cri-o://3a7bc4e2c2d03c790ca77f4b3b80ffa22b797be90efca42f365c1a0c65efd1dd" gracePeriod=30
Oct 30 12:36:41.956721 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:41.956697    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:36:42.059965 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.059907    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-certificates\") pod \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") "
Oct 30 12:36:42.059965 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.059946    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-bound-sa-token\") pod \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") "
Oct 30 12:36:42.060137 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.059974    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") pod \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") "
Oct 30 12:36:42.060137 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.060003    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-trusted-ca\") pod \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") "
Oct 30 12:36:42.060137 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.060031    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-installation-pull-secrets\") pod \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") "
Oct 30 12:36:42.060137 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.060104    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-kube-api-access-k2v9l\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-kube-api-access-k2v9l\") pod \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") "
Oct 30 12:36:42.060340 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.060149    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-ca-trust-extracted\") pod \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") "
Oct 30 12:36:42.060340 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.060202    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-image-registry-private-configuration\") pod \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\" (UID: \"bbb14793-e4b1-4912-9ce2-008cba7ce1d4\") "
Oct 30 12:36:42.060444 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.060388    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-certificates" pod "bbb14793-e4b1-4912-9ce2-008cba7ce1d4" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:36:42.060558 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.060536    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-trusted-ca" pod "bbb14793-e4b1-4912-9ce2-008cba7ce1d4" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:36:42.062873 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.062844    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-kube-api-access-k2v9l" pod "bbb14793-e4b1-4912-9ce2-008cba7ce1d4" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4"). InnerVolumeSpecName "kube-api-access-k2v9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:36:42.062998 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.062849    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-bound-sa-token" pod "bbb14793-e4b1-4912-9ce2-008cba7ce1d4" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:36:42.062998 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.062957    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls" pod "bbb14793-e4b1-4912-9ce2-008cba7ce1d4" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:36:42.062998 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.062957    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-installation-pull-secrets" pod "bbb14793-e4b1-4912-9ce2-008cba7ce1d4" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:36:42.063102 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.063041    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-image-registry-private-configuration" pod "bbb14793-e4b1-4912-9ce2-008cba7ce1d4" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:36:42.067481 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.067456    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-ca-trust-extracted" pod "bbb14793-e4b1-4912-9ce2-008cba7ce1d4" (UID: "bbb14793-e4b1-4912-9ce2-008cba7ce1d4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:36:42.161070 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.161034    4945 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-image-registry-private-configuration\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:36:42.161070 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.161068    4945 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-certificates\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:36:42.161070 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.161080    4945 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-bound-sa-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:36:42.161244 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.161090    4945 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-registry-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:36:42.161244 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.161098    4945 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-trusted-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:36:42.161244 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.161106    4945 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-installation-pull-secrets\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:36:42.161244 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.161114    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2v9l\" (UniqueName: \"kubernetes.io/projected/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-kube-api-access-k2v9l\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:36:42.161244 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.161123    4945 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bbb14793-e4b1-4912-9ce2-008cba7ce1d4-ca-trust-extracted\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:36:42.817310 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.817279    4945 generic.go:356] "Generic (PLEG): container finished" podID="bbb14793-e4b1-4912-9ce2-008cba7ce1d4" containerID="3a7bc4e2c2d03c790ca77f4b3b80ffa22b797be90efca42f365c1a0c65efd1dd" exitCode=0
Oct 30 12:36:42.817718 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.817327    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f558d755b-bj4vz"
Oct 30 12:36:42.817718 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.817328    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f558d755b-bj4vz" event={"ID":"bbb14793-e4b1-4912-9ce2-008cba7ce1d4","Type":"ContainerDied","Data":"3a7bc4e2c2d03c790ca77f4b3b80ffa22b797be90efca42f365c1a0c65efd1dd"}
Oct 30 12:36:42.817718 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.817451    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f558d755b-bj4vz" event={"ID":"bbb14793-e4b1-4912-9ce2-008cba7ce1d4","Type":"ContainerDied","Data":"31268150c1b1a09fda8387ec5da6dbd041507fdc60666d9d037fbafa577733e1"}
Oct 30 12:36:42.817718 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.817475    4945 scope.go:122] "RemoveContainer" containerID="3a7bc4e2c2d03c790ca77f4b3b80ffa22b797be90efca42f365c1a0c65efd1dd"
Oct 30 12:36:42.828034 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.828012    4945 scope.go:122] "RemoveContainer" containerID="3a7bc4e2c2d03c790ca77f4b3b80ffa22b797be90efca42f365c1a0c65efd1dd"
Oct 30 12:36:42.828321 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:42.828300    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a7bc4e2c2d03c790ca77f4b3b80ffa22b797be90efca42f365c1a0c65efd1dd\": container with ID starting with 3a7bc4e2c2d03c790ca77f4b3b80ffa22b797be90efca42f365c1a0c65efd1dd not found: ID does not exist" containerID="3a7bc4e2c2d03c790ca77f4b3b80ffa22b797be90efca42f365c1a0c65efd1dd"
Oct 30 12:36:42.828394 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.828332    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a7bc4e2c2d03c790ca77f4b3b80ffa22b797be90efca42f365c1a0c65efd1dd"} err="failed to get container status \"3a7bc4e2c2d03c790ca77f4b3b80ffa22b797be90efca42f365c1a0c65efd1dd\": rpc error: code = NotFound desc = could not find container \"3a7bc4e2c2d03c790ca77f4b3b80ffa22b797be90efca42f365c1a0c65efd1dd\": container with ID starting with 3a7bc4e2c2d03c790ca77f4b3b80ffa22b797be90efca42f365c1a0c65efd1dd not found: ID does not exist"
Oct 30 12:36:42.878202 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.878178    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f558d755b-bj4vz"]
Oct 30 12:36:42.882016 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:42.881989    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6f558d755b-bj4vz"]
Oct 30 12:36:43.546244 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:36:43.546194    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bbb14793-e4b1-4912-9ce2-008cba7ce1d4" path="/var/lib/kubelet/pods/bbb14793-e4b1-4912-9ce2-008cba7ce1d4/volumes"
Oct 30 12:36:56.189100 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:36:56.189063    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:37:00.887534 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:00.887458    4945 generic.go:356] "Generic (PLEG): container finished" podID="60e62efb-7c65-41f3-87d4-0805eec8f7e9" containerID="fe6b7ed38e8f251060a656eb1b02f153d53fa334a8866792f862087df17121e4" exitCode=0
Oct 30 12:37:00.887534 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:00.887521    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-m6xzg" event={"ID":"60e62efb-7c65-41f3-87d4-0805eec8f7e9","Type":"ContainerDied","Data":"fe6b7ed38e8f251060a656eb1b02f153d53fa334a8866792f862087df17121e4"}
Oct 30 12:37:00.887979 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:00.887817    4945 scope.go:122] "RemoveContainer" containerID="fe6b7ed38e8f251060a656eb1b02f153d53fa334a8866792f862087df17121e4"
Oct 30 12:37:01.894121 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:01.894089    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-m6xzg" event={"ID":"60e62efb-7c65-41f3-87d4-0805eec8f7e9","Type":"ContainerStarted","Data":"2107673de60c99d719a2170ed655d6fe73b82f98aa0f7ce4b61610972462e937"}
Oct 30 12:37:01.895683 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:01.895659    4945 generic.go:356] "Generic (PLEG): container finished" podID="aba04aa0-2bee-47e2-9f36-8790309d6e06" containerID="5a4a3465b092e40effab90d846efc344d30c5628a10a84d026ec220db03afbd3" exitCode=0
Oct 30 12:37:01.895796 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:01.895715    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j" event={"ID":"aba04aa0-2bee-47e2-9f36-8790309d6e06","Type":"ContainerDied","Data":"5a4a3465b092e40effab90d846efc344d30c5628a10a84d026ec220db03afbd3"}
Oct 30 12:37:01.895974 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:01.895961    4945 scope.go:122] "RemoveContainer" containerID="5a4a3465b092e40effab90d846efc344d30c5628a10a84d026ec220db03afbd3"
Oct 30 12:37:02.901259 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:02.901219    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-x976j" event={"ID":"aba04aa0-2bee-47e2-9f36-8790309d6e06","Type":"ContainerStarted","Data":"674349d0b737d5b628b3383b51d01b27c17bef27862f01d8b4f2845beae84f49"}
Oct 30 12:37:03.284949 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.284919    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-846cf5b65-gllvj_d81730d3-53ed-40b8-a512-348fc042f821/router/0.log"
Oct 30 12:37:03.439102 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.439062    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t5j98_10337f54-95e9-42cd-a73f-16817fc1cb96/serve-healthcheck-canary/0.log"
Oct 30 12:37:03.569712 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.569610    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-57b7896669-ndwsr" podUID="cc28fe61-b116-4a8b-ad04-123d62365a02" containerName="registry" containerID="cri-o://6cc54aec1b1104b8c2d3407f353d2eac0a4bb74889e540761b152a9292075dde" gracePeriod=30
Oct 30 12:37:03.813888 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.813867    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:37:03.906648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.906551    4945 generic.go:356] "Generic (PLEG): container finished" podID="cc28fe61-b116-4a8b-ad04-123d62365a02" containerID="6cc54aec1b1104b8c2d3407f353d2eac0a4bb74889e540761b152a9292075dde" exitCode=0
Oct 30 12:37:03.906648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.906593    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57b7896669-ndwsr"
Oct 30 12:37:03.906648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.906629    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57b7896669-ndwsr" event={"ID":"cc28fe61-b116-4a8b-ad04-123d62365a02","Type":"ContainerDied","Data":"6cc54aec1b1104b8c2d3407f353d2eac0a4bb74889e540761b152a9292075dde"}
Oct 30 12:37:03.907138 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.906684    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57b7896669-ndwsr" event={"ID":"cc28fe61-b116-4a8b-ad04-123d62365a02","Type":"ContainerDied","Data":"913d62efb8e205efcfa16791e0fe86f76b779ed9a9444ad68319b6b7f3b4be29"}
Oct 30 12:37:03.907138 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.906713    4945 scope.go:122] "RemoveContainer" containerID="6cc54aec1b1104b8c2d3407f353d2eac0a4bb74889e540761b152a9292075dde"
Oct 30 12:37:03.914720 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.914698    4945 scope.go:122] "RemoveContainer" containerID="6cc54aec1b1104b8c2d3407f353d2eac0a4bb74889e540761b152a9292075dde"
Oct 30 12:37:03.914992 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:37:03.914971    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cc54aec1b1104b8c2d3407f353d2eac0a4bb74889e540761b152a9292075dde\": container with ID starting with 6cc54aec1b1104b8c2d3407f353d2eac0a4bb74889e540761b152a9292075dde not found: ID does not exist" containerID="6cc54aec1b1104b8c2d3407f353d2eac0a4bb74889e540761b152a9292075dde"
Oct 30 12:37:03.915038 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.915002    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cc54aec1b1104b8c2d3407f353d2eac0a4bb74889e540761b152a9292075dde"} err="failed to get container status \"6cc54aec1b1104b8c2d3407f353d2eac0a4bb74889e540761b152a9292075dde\": rpc error: code = NotFound desc = could not find container \"6cc54aec1b1104b8c2d3407f353d2eac0a4bb74889e540761b152a9292075dde\": container with ID starting with 6cc54aec1b1104b8c2d3407f353d2eac0a4bb74889e540761b152a9292075dde not found: ID does not exist"
Oct 30 12:37:03.922890 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.922870    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") pod \"cc28fe61-b116-4a8b-ad04-123d62365a02\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") "
Oct 30 12:37:03.922970 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.922900    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc28fe61-b116-4a8b-ad04-123d62365a02-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc28fe61-b116-4a8b-ad04-123d62365a02-ca-trust-extracted\") pod \"cc28fe61-b116-4a8b-ad04-123d62365a02\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") "
Oct 30 12:37:03.922970 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.922920    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-trusted-ca\") pod \"cc28fe61-b116-4a8b-ad04-123d62365a02\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") "
Oct 30 12:37:03.923034 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.922966    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-bound-sa-token\") pod \"cc28fe61-b116-4a8b-ad04-123d62365a02\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") "
Oct 30 12:37:03.923034 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.923009    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-installation-pull-secrets\") pod \"cc28fe61-b116-4a8b-ad04-123d62365a02\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") "
Oct 30 12:37:03.923097 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.923061    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-kube-api-access-2d7qp\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-kube-api-access-2d7qp\") pod \"cc28fe61-b116-4a8b-ad04-123d62365a02\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") "
Oct 30 12:37:03.923141 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.923102    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-image-registry-private-configuration\") pod \"cc28fe61-b116-4a8b-ad04-123d62365a02\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") "
Oct 30 12:37:03.923194 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.923151    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-certificates\") pod \"cc28fe61-b116-4a8b-ad04-123d62365a02\" (UID: \"cc28fe61-b116-4a8b-ad04-123d62365a02\") "
Oct 30 12:37:03.923431 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.923404    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-trusted-ca" pod "cc28fe61-b116-4a8b-ad04-123d62365a02" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:37:03.923739 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.923712    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-certificates" pod "cc28fe61-b116-4a8b-ad04-123d62365a02" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:37:03.925291 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.925269    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-bound-sa-token" pod "cc28fe61-b116-4a8b-ad04-123d62365a02" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:37:03.925510 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.925487    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls" pod "cc28fe61-b116-4a8b-ad04-123d62365a02" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:37:03.925586 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.925553    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-installation-pull-secrets" pod "cc28fe61-b116-4a8b-ad04-123d62365a02" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:37:03.925721 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.925702    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-image-registry-private-configuration" pod "cc28fe61-b116-4a8b-ad04-123d62365a02" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:37:03.925946 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.925916    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-kube-api-access-2d7qp" pod "cc28fe61-b116-4a8b-ad04-123d62365a02" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02"). InnerVolumeSpecName "kube-api-access-2d7qp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:37:03.929788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:03.929731    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc28fe61-b116-4a8b-ad04-123d62365a02-ca-trust-extracted" pod "cc28fe61-b116-4a8b-ad04-123d62365a02" (UID: "cc28fe61-b116-4a8b-ad04-123d62365a02"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:37:04.024199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:04.024173    4945 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-certificates\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:37:04.024199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:04.024197    4945 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-registry-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:37:04.024348 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:04.024206    4945 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc28fe61-b116-4a8b-ad04-123d62365a02-ca-trust-extracted\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:37:04.024348 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:04.024216    4945 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc28fe61-b116-4a8b-ad04-123d62365a02-trusted-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:37:04.024348 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:04.024226    4945 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-bound-sa-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:37:04.024348 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:04.024235    4945 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-installation-pull-secrets\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:37:04.024348 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:04.024243    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2d7qp\" (UniqueName: \"kubernetes.io/projected/cc28fe61-b116-4a8b-ad04-123d62365a02-kube-api-access-2d7qp\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:37:04.024348 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:04.024253    4945 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc28fe61-b116-4a8b-ad04-123d62365a02-image-registry-private-configuration\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:37:04.248633 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:04.248589    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57b7896669-ndwsr"]
Oct 30 12:37:04.259785 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:04.259762    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-57b7896669-ndwsr"]
Oct 30 12:37:05.545868 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:05.545837    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc28fe61-b116-4a8b-ad04-123d62365a02" path="/var/lib/kubelet/pods/cc28fe61-b116-4a8b-ad04-123d62365a02/volumes"
Oct 30 12:37:21.976686 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:21.976650    4945 generic.go:356] "Generic (PLEG): container finished" podID="4f13aca3-562e-406b-a154-8755b1bf7d15" containerID="12b8abefe8eed3bb0629ff0eb5eaf5b1bdeb68462c185ed1b373600bd9b8a13b" exitCode=0
Oct 30 12:37:21.977092 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:21.976729    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w" event={"ID":"4f13aca3-562e-406b-a154-8755b1bf7d15","Type":"ContainerDied","Data":"12b8abefe8eed3bb0629ff0eb5eaf5b1bdeb68462c185ed1b373600bd9b8a13b"}
Oct 30 12:37:21.977092 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:21.977026    4945 scope.go:122] "RemoveContainer" containerID="12b8abefe8eed3bb0629ff0eb5eaf5b1bdeb68462c185ed1b373600bd9b8a13b"
Oct 30 12:37:22.982581 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:37:22.982549    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-b2p4w" event={"ID":"4f13aca3-562e-406b-a154-8755b1bf7d15","Type":"ContainerStarted","Data":"233c4c160352332c876ab5e02356bc14729a08cf8e6adc84b6a206815310b02a"}
Oct 30 12:37:26.189495 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:37:26.189457    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:37:56.189669 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:37:56.189628    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:38:26.190105 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:38:26.190017    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:38:56.189802 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:38:56.189767    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:39:23.346793 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.346750    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"]
Oct 30 12:39:23.347267 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.347141    4945 cpu_manager.go:414] "containerMap: removing container" podUID="bbb14793-e4b1-4912-9ce2-008cba7ce1d4" containerName="registry"
Oct 30 12:39:23.347267 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.347158    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbb14793-e4b1-4912-9ce2-008cba7ce1d4" containerName="registry"
Oct 30 12:39:23.347267 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.347184    4945 cpu_manager.go:414] "containerMap: removing container" podUID="cc28fe61-b116-4a8b-ad04-123d62365a02" containerName="registry"
Oct 30 12:39:23.347267 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.347190    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc28fe61-b116-4a8b-ad04-123d62365a02" containerName="registry"
Oct 30 12:39:23.347267 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.347235    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc28fe61-b116-4a8b-ad04-123d62365a02" containerName="registry"
Oct 30 12:39:23.347267 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.347244    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="bbb14793-e4b1-4912-9ce2-008cba7ce1d4" containerName="registry"
Oct 30 12:39:23.353053 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.353029    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"
Oct 30 12:39:23.381482 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.381456    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 12:39:23.381624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.381508    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"fdb36248658848c45b2cd30265e4aa38f5747d-4cc5d9d1-dockercfg-x9g9z\""
Oct 30 12:39:23.381624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.381521    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 12:39:23.382726 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.382702    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"]
Oct 30 12:39:23.402854 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.402829    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/add3deca-26ee-4b46-9877-68ec3c42fc73-util\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l\" (UID: \"add3deca-26ee-4b46-9877-68ec3c42fc73\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"
Oct 30 12:39:23.402960 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.402877    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/add3deca-26ee-4b46-9877-68ec3c42fc73-bundle\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l\" (UID: \"add3deca-26ee-4b46-9877-68ec3c42fc73\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"
Oct 30 12:39:23.403010 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.402955    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8f8fl\" (UniqueName: \"kubernetes.io/projected/add3deca-26ee-4b46-9877-68ec3c42fc73-kube-api-access-8f8fl\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l\" (UID: \"add3deca-26ee-4b46-9877-68ec3c42fc73\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"
Oct 30 12:39:23.437080 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.437055    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"]
Oct 30 12:39:23.440428 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.440411    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"
Oct 30 12:39:23.458816 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.458795    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"]
Oct 30 12:39:23.469931 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.469913    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-s8hgx\""
Oct 30 12:39:23.503662 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.503637    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/add3deca-26ee-4b46-9877-68ec3c42fc73-util\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l\" (UID: \"add3deca-26ee-4b46-9877-68ec3c42fc73\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"
Oct 30 12:39:23.503764 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.503672    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk\" (UID: \"d70a5f1f-3650-4af3-ad6d-6ee96e50489e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"
Oct 30 12:39:23.503764 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.503694    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/add3deca-26ee-4b46-9877-68ec3c42fc73-bundle\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l\" (UID: \"add3deca-26ee-4b46-9877-68ec3c42fc73\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"
Oct 30 12:39:23.503875 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.503819    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8f8fl\" (UniqueName: \"kubernetes.io/projected/add3deca-26ee-4b46-9877-68ec3c42fc73-kube-api-access-8f8fl\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l\" (UID: \"add3deca-26ee-4b46-9877-68ec3c42fc73\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"
Oct 30 12:39:23.503939 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.503872    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssmvk\" (UniqueName: \"kubernetes.io/projected/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-kube-api-access-ssmvk\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk\" (UID: \"d70a5f1f-3650-4af3-ad6d-6ee96e50489e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"
Oct 30 12:39:23.503939 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.503907    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk\" (UID: \"d70a5f1f-3650-4af3-ad6d-6ee96e50489e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"
Oct 30 12:39:23.504117 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.504094    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/add3deca-26ee-4b46-9877-68ec3c42fc73-bundle\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l\" (UID: \"add3deca-26ee-4b46-9877-68ec3c42fc73\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"
Oct 30 12:39:23.504117 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.504110    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/add3deca-26ee-4b46-9877-68ec3c42fc73-util\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l\" (UID: \"add3deca-26ee-4b46-9877-68ec3c42fc73\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"
Oct 30 12:39:23.530037 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.530014    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8f8fl\" (UniqueName: \"kubernetes.io/projected/add3deca-26ee-4b46-9877-68ec3c42fc73-kube-api-access-8f8fl\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l\" (UID: \"add3deca-26ee-4b46-9877-68ec3c42fc73\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"
Oct 30 12:39:23.604961 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.604908    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk\" (UID: \"d70a5f1f-3650-4af3-ad6d-6ee96e50489e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"
Oct 30 12:39:23.604961 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.604948    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssmvk\" (UniqueName: \"kubernetes.io/projected/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-kube-api-access-ssmvk\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk\" (UID: \"d70a5f1f-3650-4af3-ad6d-6ee96e50489e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"
Oct 30 12:39:23.605087 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.604966    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk\" (UID: \"d70a5f1f-3650-4af3-ad6d-6ee96e50489e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"
Oct 30 12:39:23.605323 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.605304    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk\" (UID: \"d70a5f1f-3650-4af3-ad6d-6ee96e50489e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"
Oct 30 12:39:23.605362 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.605321    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk\" (UID: \"d70a5f1f-3650-4af3-ad6d-6ee96e50489e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"
Oct 30 12:39:23.623377 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.623349    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssmvk\" (UniqueName: \"kubernetes.io/projected/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-kube-api-access-ssmvk\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk\" (UID: \"d70a5f1f-3650-4af3-ad6d-6ee96e50489e\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"
Oct 30 12:39:23.668323 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.668293    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"
Oct 30 12:39:23.755323 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.755292    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"
Oct 30 12:39:23.964967 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:23.964935    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"]
Oct 30 12:39:23.969193 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:39:23.969170    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadd3deca_26ee_4b46_9877_68ec3c42fc73.slice/crio-dd205635f89fcd5fee2d229579e5416315fb5c690b22aa367deb59c1ea7bb2ed WatchSource:0}: Error finding container dd205635f89fcd5fee2d229579e5416315fb5c690b22aa367deb59c1ea7bb2ed: Status 404 returned error can't find the container with id dd205635f89fcd5fee2d229579e5416315fb5c690b22aa367deb59c1ea7bb2ed
Oct 30 12:39:24.045140 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:24.045082    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"]
Oct 30 12:39:24.049409 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:39:24.049383    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd70a5f1f_3650_4af3_ad6d_6ee96e50489e.slice/crio-fe3940400cbb37a0eff4634b11bb10c12acd31f1f8eab79d6132e6eb174b5b7c WatchSource:0}: Error finding container fe3940400cbb37a0eff4634b11bb10c12acd31f1f8eab79d6132e6eb174b5b7c: Status 404 returned error can't find the container with id fe3940400cbb37a0eff4634b11bb10c12acd31f1f8eab79d6132e6eb174b5b7c
Oct 30 12:39:24.423197 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:24.423109    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk" event={"ID":"d70a5f1f-3650-4af3-ad6d-6ee96e50489e","Type":"ContainerStarted","Data":"fe3940400cbb37a0eff4634b11bb10c12acd31f1f8eab79d6132e6eb174b5b7c"}
Oct 30 12:39:24.424663 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:24.424637    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l" event={"ID":"add3deca-26ee-4b46-9877-68ec3c42fc73","Type":"ContainerStarted","Data":"dd205635f89fcd5fee2d229579e5416315fb5c690b22aa367deb59c1ea7bb2ed"}
Oct 30 12:39:25.648928 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:25.647153    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"]
Oct 30 12:39:25.651404 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:25.651058    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"
Oct 30 12:39:25.679836 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:25.679794    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-ph5qk\""
Oct 30 12:39:25.681042 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:25.681018    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"]
Oct 30 12:39:25.721680 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:25.721648    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3c33630-412a-4566-b890-04e0032d0e84-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb\" (UID: \"d3c33630-412a-4566-b890-04e0032d0e84\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"
Oct 30 12:39:25.721837 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:25.721726    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3c33630-412a-4566-b890-04e0032d0e84-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb\" (UID: \"d3c33630-412a-4566-b890-04e0032d0e84\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"
Oct 30 12:39:25.721837 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:25.721769    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlbdx\" (UniqueName: \"kubernetes.io/projected/d3c33630-412a-4566-b890-04e0032d0e84-kube-api-access-wlbdx\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb\" (UID: \"d3c33630-412a-4566-b890-04e0032d0e84\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"
Oct 30 12:39:25.822940 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:25.822895    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3c33630-412a-4566-b890-04e0032d0e84-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb\" (UID: \"d3c33630-412a-4566-b890-04e0032d0e84\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"
Oct 30 12:39:25.823108 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:25.822965    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlbdx\" (UniqueName: \"kubernetes.io/projected/d3c33630-412a-4566-b890-04e0032d0e84-kube-api-access-wlbdx\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb\" (UID: \"d3c33630-412a-4566-b890-04e0032d0e84\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"
Oct 30 12:39:25.823108 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:25.823029    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3c33630-412a-4566-b890-04e0032d0e84-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb\" (UID: \"d3c33630-412a-4566-b890-04e0032d0e84\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"
Oct 30 12:39:25.823470 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:25.823445    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3c33630-412a-4566-b890-04e0032d0e84-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb\" (UID: \"d3c33630-412a-4566-b890-04e0032d0e84\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"
Oct 30 12:39:25.823750 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:25.823726    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3c33630-412a-4566-b890-04e0032d0e84-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb\" (UID: \"d3c33630-412a-4566-b890-04e0032d0e84\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"
Oct 30 12:39:25.843103 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:25.843075    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlbdx\" (UniqueName: \"kubernetes.io/projected/d3c33630-412a-4566-b890-04e0032d0e84-kube-api-access-wlbdx\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb\" (UID: \"d3c33630-412a-4566-b890-04e0032d0e84\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"
Oct 30 12:39:25.970798 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:25.970765    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"
Oct 30 12:39:26.208042 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:39:26.208001    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:39:26.277020 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:26.276896    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"]
Oct 30 12:39:26.280822 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:39:26.280785    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c33630_412a_4566_b890_04e0032d0e84.slice/crio-4bead16419c98f20a66cb4cabcbac6d73acc3dd5446e07e85872119930c790dc WatchSource:0}: Error finding container 4bead16419c98f20a66cb4cabcbac6d73acc3dd5446e07e85872119930c790dc: Status 404 returned error can't find the container with id 4bead16419c98f20a66cb4cabcbac6d73acc3dd5446e07e85872119930c790dc
Oct 30 12:39:26.435842 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:26.435793    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb" event={"ID":"d3c33630-412a-4566-b890-04e0032d0e84","Type":"ContainerStarted","Data":"4bead16419c98f20a66cb4cabcbac6d73acc3dd5446e07e85872119930c790dc"}
Oct 30 12:39:26.724835 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:26.724800    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"]
Oct 30 12:39:26.728542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:26.728517    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"
Oct 30 12:39:26.747645 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:26.747532    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"]
Oct 30 12:39:26.757750 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:26.757721    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-z7mqq\""
Oct 30 12:39:26.830867 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:26.830828    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8snh2\" (UniqueName: \"kubernetes.io/projected/89064220-965d-4d1e-be0d-69e90ddea2fd-kube-api-access-8snh2\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz\" (UID: \"89064220-965d-4d1e-be0d-69e90ddea2fd\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"
Oct 30 12:39:26.831053 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:26.830915    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89064220-965d-4d1e-be0d-69e90ddea2fd-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz\" (UID: \"89064220-965d-4d1e-be0d-69e90ddea2fd\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"
Oct 30 12:39:26.831053 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:26.830945    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89064220-965d-4d1e-be0d-69e90ddea2fd-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz\" (UID: \"89064220-965d-4d1e-be0d-69e90ddea2fd\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"
Oct 30 12:39:26.931950 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:26.931915    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8snh2\" (UniqueName: \"kubernetes.io/projected/89064220-965d-4d1e-be0d-69e90ddea2fd-kube-api-access-8snh2\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz\" (UID: \"89064220-965d-4d1e-be0d-69e90ddea2fd\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"
Oct 30 12:39:26.932239 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:26.932004    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89064220-965d-4d1e-be0d-69e90ddea2fd-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz\" (UID: \"89064220-965d-4d1e-be0d-69e90ddea2fd\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"
Oct 30 12:39:26.932239 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:26.932062    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89064220-965d-4d1e-be0d-69e90ddea2fd-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz\" (UID: \"89064220-965d-4d1e-be0d-69e90ddea2fd\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"
Oct 30 12:39:26.932523 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:26.932501    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89064220-965d-4d1e-be0d-69e90ddea2fd-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz\" (UID: \"89064220-965d-4d1e-be0d-69e90ddea2fd\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"
Oct 30 12:39:26.932594 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:26.932572    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89064220-965d-4d1e-be0d-69e90ddea2fd-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz\" (UID: \"89064220-965d-4d1e-be0d-69e90ddea2fd\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"
Oct 30 12:39:26.960526 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:26.960495    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8snh2\" (UniqueName: \"kubernetes.io/projected/89064220-965d-4d1e-be0d-69e90ddea2fd-kube-api-access-8snh2\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz\" (UID: \"89064220-965d-4d1e-be0d-69e90ddea2fd\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"
Oct 30 12:39:27.048853 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:27.048771    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"
Oct 30 12:39:27.377102 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:27.377018    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"]
Oct 30 12:39:27.381540 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:39:27.381505    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89064220_965d_4d1e_be0d_69e90ddea2fd.slice/crio-ef07568302f1fdd392efa88e4cdd6e33cac7d093f422aab8eb18662ac9eba019 WatchSource:0}: Error finding container ef07568302f1fdd392efa88e4cdd6e33cac7d093f422aab8eb18662ac9eba019: Status 404 returned error can't find the container with id ef07568302f1fdd392efa88e4cdd6e33cac7d093f422aab8eb18662ac9eba019
Oct 30 12:39:27.445054 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:27.445020    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz" event={"ID":"89064220-965d-4d1e-be0d-69e90ddea2fd","Type":"ContainerStarted","Data":"ef07568302f1fdd392efa88e4cdd6e33cac7d093f422aab8eb18662ac9eba019"}
Oct 30 12:39:27.912516 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:27.912468    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"]
Oct 30 12:39:27.928686 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:27.926188    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:27.940919 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:27.940891    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"]
Oct 30 12:39:27.957116 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:27.957085    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-bn28m\""
Oct 30 12:39:27.957116 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:27.957085    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 12:39:28.041838 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.041800    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8293650f-2035-41cb-8db9-7942342fd116-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.042024 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.041850    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8293650f-2035-41cb-8db9-7942342fd116-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.042024 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.041884    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.042024 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.041946    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.042024 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.041989    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.042024 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.042022    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw6kh\" (UniqueName: \"kubernetes.io/projected/8293650f-2035-41cb-8db9-7942342fd116-kube-api-access-lw6kh\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.042269 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.042051    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.042269 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.042094    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8293650f-2035-41cb-8db9-7942342fd116-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.042269 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.042132    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.045936 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.045909    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"]
Oct 30 12:39:28.049820 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.049800    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.069489 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.069433    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-28h8m\""
Oct 30 12:39:28.071947 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.071699    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"]
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.142962    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143001    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143020    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143045    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143067    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143087    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143101    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lw6kh\" (UniqueName: \"kubernetes.io/projected/8293650f-2035-41cb-8db9-7942342fd116-kube-api-access-lw6kh\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143119    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143146    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2jcz\" (UniqueName: \"kubernetes.io/projected/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-kube-api-access-d2jcz\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143163    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8293650f-2035-41cb-8db9-7942342fd116-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143185    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143202    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143227    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143253    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143285    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8293650f-2035-41cb-8db9-7942342fd116-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143307    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8293650f-2035-41cb-8db9-7942342fd116-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143338    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.143639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143364    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.144648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143705    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.144648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.143896    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.144648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.144305    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.144648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.144471    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8293650f-2035-41cb-8db9-7942342fd116-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.144868 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.144675    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.147192 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.147039    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8293650f-2035-41cb-8db9-7942342fd116-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.147321 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.147222    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.162283 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.162244    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw6kh\" (UniqueName: \"kubernetes.io/projected/8293650f-2035-41cb-8db9-7942342fd116-kube-api-access-lw6kh\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.162464 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.162438    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8293650f-2035-41cb-8db9-7942342fd116-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-wtc5x\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.244522 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.244477    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.244725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.244552    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.244725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.244593    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.244725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.244645    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.244725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.244670    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.244725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.244714    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.244980 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.244768    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2jcz\" (UniqueName: \"kubernetes.io/projected/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-kube-api-access-d2jcz\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.244980 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.244810    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.244980 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.244834    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.245273 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.245254    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.245498 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.245472    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.246981 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.246958    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.247200 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.247175    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.247421 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.247394    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.247550 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.247533    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:28.249567 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.249518    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.250767 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.250720    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.264786 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.264726    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2jcz\" (UniqueName: \"kubernetes.io/projected/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-kube-api-access-d2jcz\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.266477 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.266454    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e9dc79c-741f-465e-ba5d-3c99e6d7f812-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5\" (UID: \"6e9dc79c-741f-465e-ba5d-3c99e6d7f812\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.378632 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.378357    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:28.588315 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.588211    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"]
Oct 30 12:39:28.593888 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:39:28.593852    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8293650f_2035_41cb_8db9_7942342fd116.slice/crio-9b2d61bd088e792b2cb6292d26aa7971cec7e95f52c98b0d2da0966531993808 WatchSource:0}: Error finding container 9b2d61bd088e792b2cb6292d26aa7971cec7e95f52c98b0d2da0966531993808: Status 404 returned error can't find the container with id 9b2d61bd088e792b2cb6292d26aa7971cec7e95f52c98b0d2da0966531993808
Oct 30 12:39:28.690366 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:28.690326    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"]
Oct 30 12:39:28.697174 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:39:28.697141    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e9dc79c_741f_465e_ba5d_3c99e6d7f812.slice/crio-9c1abfe38a36495e82c9365f735bc8d374d18a44a924f8239c65824b1bd6f564 WatchSource:0}: Error finding container 9c1abfe38a36495e82c9365f735bc8d374d18a44a924f8239c65824b1bd6f564: Status 404 returned error can't find the container with id 9c1abfe38a36495e82c9365f735bc8d374d18a44a924f8239c65824b1bd6f564
Oct 30 12:39:29.414566 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:29.414512    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"]
Oct 30 12:39:29.419377 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:29.419348    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"
Oct 30 12:39:29.446617 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:29.446525    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"]
Oct 30 12:39:29.475343 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:29.475312    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5" event={"ID":"6e9dc79c-741f-465e-ba5d-3c99e6d7f812","Type":"ContainerStarted","Data":"9c1abfe38a36495e82c9365f735bc8d374d18a44a924f8239c65824b1bd6f564"}
Oct 30 12:39:29.478234 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:29.478202    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x" event={"ID":"8293650f-2035-41cb-8db9-7942342fd116","Type":"ContainerStarted","Data":"9b2d61bd088e792b2cb6292d26aa7971cec7e95f52c98b0d2da0966531993808"}
Oct 30 12:39:29.567118 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:29.566835    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6462\" (UniqueName: \"kubernetes.io/projected/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-kube-api-access-t6462\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz\" (UID: \"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"
Oct 30 12:39:29.567118 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:29.566957    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-bundle\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz\" (UID: \"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"
Oct 30 12:39:29.567118 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:29.566994    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-util\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz\" (UID: \"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"
Oct 30 12:39:29.669497 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:29.668329    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-bundle\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz\" (UID: \"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"
Oct 30 12:39:29.669497 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:29.668385    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-util\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz\" (UID: \"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"
Oct 30 12:39:29.669497 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:29.668424    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6462\" (UniqueName: \"kubernetes.io/projected/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-kube-api-access-t6462\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz\" (UID: \"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"
Oct 30 12:39:29.669497 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:29.669168    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-bundle\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz\" (UID: \"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"
Oct 30 12:39:29.669497 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:29.669412    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-util\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz\" (UID: \"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"
Oct 30 12:39:29.690165 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:29.690128    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6462\" (UniqueName: \"kubernetes.io/projected/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-kube-api-access-t6462\") pod \"fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz\" (UID: \"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2\") " pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"
Oct 30 12:39:29.745225 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:29.745192    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"
Oct 30 12:39:30.128240 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:30.128167    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"]
Oct 30 12:39:30.485900 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:30.485817    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz" event={"ID":"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2","Type":"ContainerStarted","Data":"462ebac4fa67557848c045679b466f26368bb12fc69fb2f707370e4861fbb9f1"}
Oct 30 12:39:33.515121 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:33.515081    4945 generic.go:356] "Generic (PLEG): container finished" podID="d70a5f1f-3650-4af3-ad6d-6ee96e50489e" containerID="f0ceefe7e8a9f32de48c9d651715d5ce3c941b9cd7d77eb47491b83236e3cbfe" exitCode=0
Oct 30 12:39:33.515570 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:33.515209    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk" event={"ID":"d70a5f1f-3650-4af3-ad6d-6ee96e50489e","Type":"ContainerDied","Data":"f0ceefe7e8a9f32de48c9d651715d5ce3c941b9cd7d77eb47491b83236e3cbfe"}
Oct 30 12:39:33.520276 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:33.520249    4945 generic.go:356] "Generic (PLEG): container finished" podID="d3c33630-412a-4566-b890-04e0032d0e84" containerID="1b5235d174f855238e63dcba6f5fac31dbe28ffb07df2337349bdac61e81eaa1" exitCode=0
Oct 30 12:39:33.520371 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:33.520330    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb" event={"ID":"d3c33630-412a-4566-b890-04e0032d0e84","Type":"ContainerDied","Data":"1b5235d174f855238e63dcba6f5fac31dbe28ffb07df2337349bdac61e81eaa1"}
Oct 30 12:39:33.524025 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:33.523917    4945 generic.go:356] "Generic (PLEG): container finished" podID="add3deca-26ee-4b46-9877-68ec3c42fc73" containerID="e99a3b94d0d456be74ff80b1ee1304e2ae695726b1ffbd45d2e1245881328be8" exitCode=0
Oct 30 12:39:33.524025 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:33.523988    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l" event={"ID":"add3deca-26ee-4b46-9877-68ec3c42fc73","Type":"ContainerDied","Data":"e99a3b94d0d456be74ff80b1ee1304e2ae695726b1ffbd45d2e1245881328be8"}
Oct 30 12:39:34.215441 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.215399    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:39:34.215557 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.215517    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:39:34.215627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.215560    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:39:34.295072 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.293884    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:39:34.295186 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.295130    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:39:34.295186 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.295177    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:39:34.533149 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.533114    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5" event={"ID":"6e9dc79c-741f-465e-ba5d-3c99e6d7f812","Type":"ContainerStarted","Data":"5f5f14d7c7755265411a0c0b51a40d79576fa3147f105660d99c0ae18f3e6687"}
Oct 30 12:39:34.536225 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.535789    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x" event={"ID":"8293650f-2035-41cb-8db9-7942342fd116","Type":"ContainerStarted","Data":"bd6c28a719533806985d76d39806e65acc3e2f42292f2897100320590bfeb490"}
Oct 30 12:39:34.537638 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.537618    4945 generic.go:356] "Generic (PLEG): container finished" podID="89064220-965d-4d1e-be0d-69e90ddea2fd" containerID="9d77908cf59188a6146e0acf5666b3d91dcc7f1d383ea1ffc846a2bf5cc5840c" exitCode=0
Oct 30 12:39:34.537730 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.537672    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz" event={"ID":"89064220-965d-4d1e-be0d-69e90ddea2fd","Type":"ContainerDied","Data":"9d77908cf59188a6146e0acf5666b3d91dcc7f1d383ea1ffc846a2bf5cc5840c"}
Oct 30 12:39:34.574681 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.573583    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5" podStartSLOduration=0.981827422 podStartE2EDuration="6.573572045s" totalImagesPullingTime="5.591744623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:39:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:39:34.573260572 +0000 UTC m=+219.151209663" watchObservedRunningTime="2026-10-30 12:39:34.573572045 +0000 UTC m=+219.151521133"
Oct 30 12:39:34.614629 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.614562    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x" podStartSLOduration=1.9971186300000001 podStartE2EDuration="7.614541853s" totalImagesPullingTime="5.617423223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:39:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:39:34.613332664 +0000 UTC m=+219.191281754" watchObservedRunningTime="2026-10-30 12:39:34.614541853 +0000 UTC m=+219.192490944"
Oct 30 12:39:34.656503 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.654542    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"]
Oct 30 12:39:34.657941 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.657919    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"
Oct 30 12:39:34.676393 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.676363    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"]
Oct 30 12:39:34.822549 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.822475    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e66f60b4-ef15-4c45-bab3-873f4351e582-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x\" (UID: \"e66f60b4-ef15-4c45-bab3-873f4351e582\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"
Oct 30 12:39:34.822549 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.822534    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv6mh\" (UniqueName: \"kubernetes.io/projected/e66f60b4-ef15-4c45-bab3-873f4351e582-kube-api-access-tv6mh\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x\" (UID: \"e66f60b4-ef15-4c45-bab3-873f4351e582\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"
Oct 30 12:39:34.822784 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.822704    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e66f60b4-ef15-4c45-bab3-873f4351e582-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x\" (UID: \"e66f60b4-ef15-4c45-bab3-873f4351e582\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"
Oct 30 12:39:34.923976 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.923940    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv6mh\" (UniqueName: \"kubernetes.io/projected/e66f60b4-ef15-4c45-bab3-873f4351e582-kube-api-access-tv6mh\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x\" (UID: \"e66f60b4-ef15-4c45-bab3-873f4351e582\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"
Oct 30 12:39:34.923976 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.923988    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e66f60b4-ef15-4c45-bab3-873f4351e582-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x\" (UID: \"e66f60b4-ef15-4c45-bab3-873f4351e582\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"
Oct 30 12:39:34.924185 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.924064    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e66f60b4-ef15-4c45-bab3-873f4351e582-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x\" (UID: \"e66f60b4-ef15-4c45-bab3-873f4351e582\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"
Oct 30 12:39:34.924440 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.924421    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e66f60b4-ef15-4c45-bab3-873f4351e582-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x\" (UID: \"e66f60b4-ef15-4c45-bab3-873f4351e582\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"
Oct 30 12:39:34.925017 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.924991    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e66f60b4-ef15-4c45-bab3-873f4351e582-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x\" (UID: \"e66f60b4-ef15-4c45-bab3-873f4351e582\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"
Oct 30 12:39:34.943894 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.943868    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv6mh\" (UniqueName: \"kubernetes.io/projected/e66f60b4-ef15-4c45-bab3-873f4351e582-kube-api-access-tv6mh\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x\" (UID: \"e66f60b4-ef15-4c45-bab3-873f4351e582\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"
Oct 30 12:39:34.977802 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:34.977778    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"
Oct 30 12:39:35.248363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:35.248328    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:35.254147 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:35.254124    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:35.281262 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:35.281191    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"]
Oct 30 12:39:35.286353 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:39:35.286327    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode66f60b4_ef15_4c45_bab3_873f4351e582.slice/crio-6103a706c9d5c1898ca82b8291add1d85390684a98e03abf2a6b011c40e52daa WatchSource:0}: Error finding container 6103a706c9d5c1898ca82b8291add1d85390684a98e03abf2a6b011c40e52daa: Status 404 returned error can't find the container with id 6103a706c9d5c1898ca82b8291add1d85390684a98e03abf2a6b011c40e52daa
Oct 30 12:39:35.379371 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:35.379348    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:35.384750 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:35.384728    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:35.548903 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:35.548866    4945 generic.go:356] "Generic (PLEG): container finished" podID="89064220-965d-4d1e-be0d-69e90ddea2fd" containerID="dae77e500d35eaa9e75ed459697181e1289fe6c4c3c48258267aa0e91f032952" exitCode=0
Oct 30 12:39:35.551171 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:35.551104    4945 generic.go:356] "Generic (PLEG): container finished" podID="e66f60b4-ef15-4c45-bab3-873f4351e582" containerID="619f6d50ed4b77068136da79a395290aba427373cae1214b5497882abd7b80b4" exitCode=0
Oct 30 12:39:35.557159 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:35.557129    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz" event={"ID":"89064220-965d-4d1e-be0d-69e90ddea2fd","Type":"ContainerDied","Data":"dae77e500d35eaa9e75ed459697181e1289fe6c4c3c48258267aa0e91f032952"}
Oct 30 12:39:35.557255 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:35.557187    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:35.557255 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:35.557213    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:35.557255 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:35.557234    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x" event={"ID":"e66f60b4-ef15-4c45-bab3-873f4351e582","Type":"ContainerDied","Data":"619f6d50ed4b77068136da79a395290aba427373cae1214b5497882abd7b80b4"}
Oct 30 12:39:35.557420 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:35.557257    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x" event={"ID":"e66f60b4-ef15-4c45-bab3-873f4351e582","Type":"ContainerStarted","Data":"6103a706c9d5c1898ca82b8291add1d85390684a98e03abf2a6b011c40e52daa"}
Oct 30 12:39:35.557420 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:35.557316    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:39:35.557420 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:35.557368    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5"
Oct 30 12:39:36.258410 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:39:36.258374    4945 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadd3deca_26ee_4b46_9877_68ec3c42fc73.slice/crio-23cd7c60c5ecf62641aa6185b67d729c2f0796dde1d72e9d38565c932b453dec.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 12:39:36.559344 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:36.559256    4945 generic.go:356] "Generic (PLEG): container finished" podID="edb1fef5-7b98-4348-8f6b-a0fca0dc14c2" containerID="c1490cc675e2ab598f2232174d6c81c59217cde81a6f449995872b6f9aefdcae" exitCode=0
Oct 30 12:39:36.559839 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:36.559346    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz" event={"ID":"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2","Type":"ContainerDied","Data":"c1490cc675e2ab598f2232174d6c81c59217cde81a6f449995872b6f9aefdcae"}
Oct 30 12:39:36.562990 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:36.562516    4945 generic.go:356] "Generic (PLEG): container finished" podID="add3deca-26ee-4b46-9877-68ec3c42fc73" containerID="23cd7c60c5ecf62641aa6185b67d729c2f0796dde1d72e9d38565c932b453dec" exitCode=0
Oct 30 12:39:36.562990 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:36.562627    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l" event={"ID":"add3deca-26ee-4b46-9877-68ec3c42fc73","Type":"ContainerDied","Data":"23cd7c60c5ecf62641aa6185b67d729c2f0796dde1d72e9d38565c932b453dec"}
Oct 30 12:39:37.570203 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:37.569990    4945 generic.go:356] "Generic (PLEG): container finished" podID="edb1fef5-7b98-4348-8f6b-a0fca0dc14c2" containerID="074c55311b0f3bc7fac1f0678fb67963a2173c850c3004c5eb678e570c92fe10" exitCode=0
Oct 30 12:39:37.570203 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:37.570134    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz" event={"ID":"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2","Type":"ContainerDied","Data":"074c55311b0f3bc7fac1f0678fb67963a2173c850c3004c5eb678e570c92fe10"}
Oct 30 12:39:37.573225 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:37.573183    4945 generic.go:356] "Generic (PLEG): container finished" podID="d70a5f1f-3650-4af3-ad6d-6ee96e50489e" containerID="af76249f87276e355b6e2fdbe702f9be35a7c95b88ce3ecaf1c458f254c3b13e" exitCode=0
Oct 30 12:39:37.573338 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:37.573265    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk" event={"ID":"d70a5f1f-3650-4af3-ad6d-6ee96e50489e","Type":"ContainerDied","Data":"af76249f87276e355b6e2fdbe702f9be35a7c95b88ce3ecaf1c458f254c3b13e"}
Oct 30 12:39:38.584220 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:38.583184    4945 generic.go:356] "Generic (PLEG): container finished" podID="e66f60b4-ef15-4c45-bab3-873f4351e582" containerID="9e505179583452431246410cfdba01864d18ec1d00cdc94ba0070a9029f89d6c" exitCode=0
Oct 30 12:39:38.584220 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:38.583254    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x" event={"ID":"e66f60b4-ef15-4c45-bab3-873f4351e582","Type":"ContainerDied","Data":"9e505179583452431246410cfdba01864d18ec1d00cdc94ba0070a9029f89d6c"}
Oct 30 12:39:39.594975 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:39.593991    4945 generic.go:356] "Generic (PLEG): container finished" podID="d3c33630-412a-4566-b890-04e0032d0e84" containerID="823b356457e45e0659b4381d53b085fe1b1c19ce7c179186c845cba43df3ac3d" exitCode=0
Oct 30 12:39:39.594975 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:39.594152    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb" event={"ID":"d3c33630-412a-4566-b890-04e0032d0e84","Type":"ContainerDied","Data":"823b356457e45e0659b4381d53b085fe1b1c19ce7c179186c845cba43df3ac3d"}
Oct 30 12:39:47.679214 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:47.679138    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz" event={"ID":"89064220-965d-4d1e-be0d-69e90ddea2fd","Type":"ContainerStarted","Data":"b9c1a6b95d64f421ab7858e1bf2dafc6d8190ba383cbd1795651fbcfa3d566c9"}
Oct 30 12:39:47.725965 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:47.725108    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz" podStartSLOduration=2.417553612 podStartE2EDuration="21.725093235s" totalImagesPullingTime="19.307539623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:39:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:39:47.722920394 +0000 UTC m=+232.300869486" watchObservedRunningTime="2026-10-30 12:39:47.725093235 +0000 UTC m=+232.303042323"
Oct 30 12:39:48.686796 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:48.686756    4945 generic.go:356] "Generic (PLEG): container finished" podID="d3c33630-412a-4566-b890-04e0032d0e84" containerID="01922e9e225dd8dc5cbfd99904fda7d7230efc5450818a2cb5c68673b4382efc" exitCode=0
Oct 30 12:39:48.687255 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:48.686981    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb" event={"ID":"d3c33630-412a-4566-b890-04e0032d0e84","Type":"ContainerDied","Data":"01922e9e225dd8dc5cbfd99904fda7d7230efc5450818a2cb5c68673b4382efc"}
Oct 30 12:39:48.689358 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:48.689333    4945 generic.go:356] "Generic (PLEG): container finished" podID="89064220-965d-4d1e-be0d-69e90ddea2fd" containerID="b9c1a6b95d64f421ab7858e1bf2dafc6d8190ba383cbd1795651fbcfa3d566c9" exitCode=0
Oct 30 12:39:48.689470 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:48.689403    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz" event={"ID":"89064220-965d-4d1e-be0d-69e90ddea2fd","Type":"ContainerDied","Data":"b9c1a6b95d64f421ab7858e1bf2dafc6d8190ba383cbd1795651fbcfa3d566c9"}
Oct 30 12:39:48.693297 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:48.693270    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l" event={"ID":"add3deca-26ee-4b46-9877-68ec3c42fc73","Type":"ContainerStarted","Data":"3062d8edf2e783ddf6172696c957e90aa7866bebf45c93661ecdf88d0b1592a6"}
Oct 30 12:39:48.764210 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:48.764080    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l" podStartSLOduration=2.5617063939999998 podStartE2EDuration="25.764062123s" totalImagesPullingTime="23.202355729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:39:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:39:48.763094017 +0000 UTC m=+233.341043109" watchObservedRunningTime="2026-10-30 12:39:48.764062123 +0000 UTC m=+233.342011218"
Oct 30 12:39:49.701298 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.701252    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk" event={"ID":"d70a5f1f-3650-4af3-ad6d-6ee96e50489e","Type":"ContainerStarted","Data":"b031ee2afacaec39df7054ad4dc23f12c1c5c8c61de2a67c9e6a6617910747b0"}
Oct 30 12:39:49.704121 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.704091    4945 generic.go:356] "Generic (PLEG): container finished" podID="add3deca-26ee-4b46-9877-68ec3c42fc73" containerID="3062d8edf2e783ddf6172696c957e90aa7866bebf45c93661ecdf88d0b1592a6" exitCode=0
Oct 30 12:39:49.704239 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.704123    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l" event={"ID":"add3deca-26ee-4b46-9877-68ec3c42fc73","Type":"ContainerDied","Data":"3062d8edf2e783ddf6172696c957e90aa7866bebf45c93661ecdf88d0b1592a6"}
Oct 30 12:39:49.743348 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.743128    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk" podStartSLOduration=1.8418467060000001 podStartE2EDuration="26.743113368s" totalImagesPullingTime="24.901266662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:39:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:39:49.741208984 +0000 UTC m=+234.319158074" watchObservedRunningTime="2026-10-30 12:39:49.743113368 +0000 UTC m=+234.321062451"
Oct 30 12:39:49.956008 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.955980    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"
Oct 30 12:39:49.968779 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.968752    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"
Oct 30 12:39:49.977030 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.977005    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89064220-965d-4d1e-be0d-69e90ddea2fd-util\" (UniqueName: \"kubernetes.io/empty-dir/89064220-965d-4d1e-be0d-69e90ddea2fd-util\") pod \"89064220-965d-4d1e-be0d-69e90ddea2fd\" (UID: \"89064220-965d-4d1e-be0d-69e90ddea2fd\") "
Oct 30 12:39:49.977141 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.977043    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d3c33630-412a-4566-b890-04e0032d0e84-util\" (UniqueName: \"kubernetes.io/empty-dir/d3c33630-412a-4566-b890-04e0032d0e84-util\") pod \"d3c33630-412a-4566-b890-04e0032d0e84\" (UID: \"d3c33630-412a-4566-b890-04e0032d0e84\") "
Oct 30 12:39:49.977141 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.977124    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d3c33630-412a-4566-b890-04e0032d0e84-bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3c33630-412a-4566-b890-04e0032d0e84-bundle\") pod \"d3c33630-412a-4566-b890-04e0032d0e84\" (UID: \"d3c33630-412a-4566-b890-04e0032d0e84\") "
Oct 30 12:39:49.977246 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.977163    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89064220-965d-4d1e-be0d-69e90ddea2fd-bundle\" (UniqueName: \"kubernetes.io/empty-dir/89064220-965d-4d1e-be0d-69e90ddea2fd-bundle\") pod \"89064220-965d-4d1e-be0d-69e90ddea2fd\" (UID: \"89064220-965d-4d1e-be0d-69e90ddea2fd\") "
Oct 30 12:39:49.977246 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.977201    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89064220-965d-4d1e-be0d-69e90ddea2fd-kube-api-access-8snh2\" (UniqueName: \"kubernetes.io/projected/89064220-965d-4d1e-be0d-69e90ddea2fd-kube-api-access-8snh2\") pod \"89064220-965d-4d1e-be0d-69e90ddea2fd\" (UID: \"89064220-965d-4d1e-be0d-69e90ddea2fd\") "
Oct 30 12:39:49.977246 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.977240    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3c33630-412a-4566-b890-04e0032d0e84-kube-api-access-wlbdx\" (UniqueName: \"kubernetes.io/projected/d3c33630-412a-4566-b890-04e0032d0e84-kube-api-access-wlbdx\") pod \"d3c33630-412a-4566-b890-04e0032d0e84\" (UID: \"d3c33630-412a-4566-b890-04e0032d0e84\") "
Oct 30 12:39:49.977887 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.977833    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89064220-965d-4d1e-be0d-69e90ddea2fd-bundle" pod "89064220-965d-4d1e-be0d-69e90ddea2fd" (UID: "89064220-965d-4d1e-be0d-69e90ddea2fd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:39:49.977974 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.977909    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3c33630-412a-4566-b890-04e0032d0e84-bundle" pod "d3c33630-412a-4566-b890-04e0032d0e84" (UID: "d3c33630-412a-4566-b890-04e0032d0e84"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:39:49.979971 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.979944    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89064220-965d-4d1e-be0d-69e90ddea2fd-kube-api-access-8snh2" pod "89064220-965d-4d1e-be0d-69e90ddea2fd" (UID: "89064220-965d-4d1e-be0d-69e90ddea2fd"). InnerVolumeSpecName "kube-api-access-8snh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:39:49.980338 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.980319    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3c33630-412a-4566-b890-04e0032d0e84-kube-api-access-wlbdx" pod "d3c33630-412a-4566-b890-04e0032d0e84" (UID: "d3c33630-412a-4566-b890-04e0032d0e84"). InnerVolumeSpecName "kube-api-access-wlbdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:39:49.981964 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.981943    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3c33630-412a-4566-b890-04e0032d0e84-util" pod "d3c33630-412a-4566-b890-04e0032d0e84" (UID: "d3c33630-412a-4566-b890-04e0032d0e84"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:39:49.983205 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:49.983186    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89064220-965d-4d1e-be0d-69e90ddea2fd-util" pod "89064220-965d-4d1e-be0d-69e90ddea2fd" (UID: "89064220-965d-4d1e-be0d-69e90ddea2fd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:39:50.078143 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.078070    4945 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3c33630-412a-4566-b890-04e0032d0e84-bundle\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:50.078143 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.078097    4945 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89064220-965d-4d1e-be0d-69e90ddea2fd-bundle\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:50.078143 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.078109    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8snh2\" (UniqueName: \"kubernetes.io/projected/89064220-965d-4d1e-be0d-69e90ddea2fd-kube-api-access-8snh2\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:50.078143 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.078118    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlbdx\" (UniqueName: \"kubernetes.io/projected/d3c33630-412a-4566-b890-04e0032d0e84-kube-api-access-wlbdx\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:50.078143 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.078127    4945 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89064220-965d-4d1e-be0d-69e90ddea2fd-util\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:50.078143 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.078136    4945 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3c33630-412a-4566-b890-04e0032d0e84-util\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:50.710102 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.710047    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb"
Oct 30 12:39:50.710489 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.710046    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c64wdb" event={"ID":"d3c33630-412a-4566-b890-04e0032d0e84","Type":"ContainerDied","Data":"4bead16419c98f20a66cb4cabcbac6d73acc3dd5446e07e85872119930c790dc"}
Oct 30 12:39:50.710489 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.710173    4945 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4bead16419c98f20a66cb4cabcbac6d73acc3dd5446e07e85872119930c790dc"
Oct 30 12:39:50.711956 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.711936    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz"
Oct 30 12:39:50.712071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.711963    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71625qz" event={"ID":"89064220-965d-4d1e-be0d-69e90ddea2fd","Type":"ContainerDied","Data":"ef07568302f1fdd392efa88e4cdd6e33cac7d093f422aab8eb18662ac9eba019"}
Oct 30 12:39:50.712071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.711996    4945 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef07568302f1fdd392efa88e4cdd6e33cac7d093f422aab8eb18662ac9eba019"
Oct 30 12:39:50.714020 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.713997    4945 generic.go:356] "Generic (PLEG): container finished" podID="e66f60b4-ef15-4c45-bab3-873f4351e582" containerID="e90d6d1db373358dd3f46fbdaf7caf8c04e1f0ca0117f52df5860bff2617015e" exitCode=0
Oct 30 12:39:50.714150 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.714093    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x" event={"ID":"e66f60b4-ef15-4c45-bab3-873f4351e582","Type":"ContainerDied","Data":"e90d6d1db373358dd3f46fbdaf7caf8c04e1f0ca0117f52df5860bff2617015e"}
Oct 30 12:39:50.716700 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.716673    4945 generic.go:356] "Generic (PLEG): container finished" podID="edb1fef5-7b98-4348-8f6b-a0fca0dc14c2" containerID="85908afa27171b2eac07e6c20a05f09edcec88eda47e65bd1e5510aec82c8715" exitCode=0
Oct 30 12:39:50.716801 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.716784    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz" event={"ID":"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2","Type":"ContainerDied","Data":"85908afa27171b2eac07e6c20a05f09edcec88eda47e65bd1e5510aec82c8715"}
Oct 30 12:39:50.718815 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.718791    4945 generic.go:356] "Generic (PLEG): container finished" podID="d70a5f1f-3650-4af3-ad6d-6ee96e50489e" containerID="b031ee2afacaec39df7054ad4dc23f12c1c5c8c61de2a67c9e6a6617910747b0" exitCode=0
Oct 30 12:39:50.718887 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.718850    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk" event={"ID":"d70a5f1f-3650-4af3-ad6d-6ee96e50489e","Type":"ContainerDied","Data":"b031ee2afacaec39df7054ad4dc23f12c1c5c8c61de2a67c9e6a6617910747b0"}
Oct 30 12:39:50.853698 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.853666    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"
Oct 30 12:39:50.884635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.884587    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/add3deca-26ee-4b46-9877-68ec3c42fc73-util\" (UniqueName: \"kubernetes.io/empty-dir/add3deca-26ee-4b46-9877-68ec3c42fc73-util\") pod \"add3deca-26ee-4b46-9877-68ec3c42fc73\" (UID: \"add3deca-26ee-4b46-9877-68ec3c42fc73\") "
Oct 30 12:39:50.884748 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.884648    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/add3deca-26ee-4b46-9877-68ec3c42fc73-kube-api-access-8f8fl\" (UniqueName: \"kubernetes.io/projected/add3deca-26ee-4b46-9877-68ec3c42fc73-kube-api-access-8f8fl\") pod \"add3deca-26ee-4b46-9877-68ec3c42fc73\" (UID: \"add3deca-26ee-4b46-9877-68ec3c42fc73\") "
Oct 30 12:39:50.884748 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.884710    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/add3deca-26ee-4b46-9877-68ec3c42fc73-bundle\" (UniqueName: \"kubernetes.io/empty-dir/add3deca-26ee-4b46-9877-68ec3c42fc73-bundle\") pod \"add3deca-26ee-4b46-9877-68ec3c42fc73\" (UID: \"add3deca-26ee-4b46-9877-68ec3c42fc73\") "
Oct 30 12:39:50.885330 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.885304    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/add3deca-26ee-4b46-9877-68ec3c42fc73-bundle" pod "add3deca-26ee-4b46-9877-68ec3c42fc73" (UID: "add3deca-26ee-4b46-9877-68ec3c42fc73"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:39:50.886627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.886589    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/add3deca-26ee-4b46-9877-68ec3c42fc73-kube-api-access-8f8fl" pod "add3deca-26ee-4b46-9877-68ec3c42fc73" (UID: "add3deca-26ee-4b46-9877-68ec3c42fc73"). InnerVolumeSpecName "kube-api-access-8f8fl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:39:50.891300 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.891271    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/add3deca-26ee-4b46-9877-68ec3c42fc73-util" pod "add3deca-26ee-4b46-9877-68ec3c42fc73" (UID: "add3deca-26ee-4b46-9877-68ec3c42fc73"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:39:50.985671 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.985594    4945 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/add3deca-26ee-4b46-9877-68ec3c42fc73-bundle\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:50.985671 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.985634    4945 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/add3deca-26ee-4b46-9877-68ec3c42fc73-util\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:50.985671 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:50.985643    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8f8fl\" (UniqueName: \"kubernetes.io/projected/add3deca-26ee-4b46-9877-68ec3c42fc73-kube-api-access-8f8fl\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:51.724511 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.724478    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l"
Oct 30 12:39:51.724942 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.724529    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cef2j76l" event={"ID":"add3deca-26ee-4b46-9877-68ec3c42fc73","Type":"ContainerDied","Data":"dd205635f89fcd5fee2d229579e5416315fb5c690b22aa367deb59c1ea7bb2ed"}
Oct 30 12:39:51.724942 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.724572    4945 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd205635f89fcd5fee2d229579e5416315fb5c690b22aa367deb59c1ea7bb2ed"
Oct 30 12:39:51.863304 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.863278    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"
Oct 30 12:39:51.891228 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.891197    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-bundle\" (UniqueName: \"kubernetes.io/empty-dir/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-bundle\") pod \"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2\" (UID: \"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2\") "
Oct 30 12:39:51.891346 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.891299    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-kube-api-access-t6462\" (UniqueName: \"kubernetes.io/projected/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-kube-api-access-t6462\") pod \"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2\" (UID: \"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2\") "
Oct 30 12:39:51.891420 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.891357    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-util\" (UniqueName: \"kubernetes.io/empty-dir/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-util\") pod \"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2\" (UID: \"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2\") "
Oct 30 12:39:51.891849 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.891822    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-bundle" pod "edb1fef5-7b98-4348-8f6b-a0fca0dc14c2" (UID: "edb1fef5-7b98-4348-8f6b-a0fca0dc14c2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:39:51.897557 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.897527    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-kube-api-access-t6462" pod "edb1fef5-7b98-4348-8f6b-a0fca0dc14c2" (UID: "edb1fef5-7b98-4348-8f6b-a0fca0dc14c2"). InnerVolumeSpecName "kube-api-access-t6462". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:39:51.900048 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.900021    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-util" pod "edb1fef5-7b98-4348-8f6b-a0fca0dc14c2" (UID: "edb1fef5-7b98-4348-8f6b-a0fca0dc14c2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:39:51.913859 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.913837    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"
Oct 30 12:39:51.916672 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.916655    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"
Oct 30 12:39:51.992117 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.992044    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-util\" (UniqueName: \"kubernetes.io/empty-dir/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-util\") pod \"d70a5f1f-3650-4af3-ad6d-6ee96e50489e\" (UID: \"d70a5f1f-3650-4af3-ad6d-6ee96e50489e\") "
Oct 30 12:39:51.992117 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.992096    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e66f60b4-ef15-4c45-bab3-873f4351e582-util\" (UniqueName: \"kubernetes.io/empty-dir/e66f60b4-ef15-4c45-bab3-873f4351e582-util\") pod \"e66f60b4-ef15-4c45-bab3-873f4351e582\" (UID: \"e66f60b4-ef15-4c45-bab3-873f4351e582\") "
Oct 30 12:39:51.992327 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.992124    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e66f60b4-ef15-4c45-bab3-873f4351e582-bundle\" (UniqueName: \"kubernetes.io/empty-dir/e66f60b4-ef15-4c45-bab3-873f4351e582-bundle\") pod \"e66f60b4-ef15-4c45-bab3-873f4351e582\" (UID: \"e66f60b4-ef15-4c45-bab3-873f4351e582\") "
Oct 30 12:39:51.992327 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.992151    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-bundle\" (UniqueName: \"kubernetes.io/empty-dir/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-bundle\") pod \"d70a5f1f-3650-4af3-ad6d-6ee96e50489e\" (UID: \"d70a5f1f-3650-4af3-ad6d-6ee96e50489e\") "
Oct 30 12:39:51.992327 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.992199    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e66f60b4-ef15-4c45-bab3-873f4351e582-kube-api-access-tv6mh\" (UniqueName: \"kubernetes.io/projected/e66f60b4-ef15-4c45-bab3-873f4351e582-kube-api-access-tv6mh\") pod \"e66f60b4-ef15-4c45-bab3-873f4351e582\" (UID: \"e66f60b4-ef15-4c45-bab3-873f4351e582\") "
Oct 30 12:39:51.992327 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.992235    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-kube-api-access-ssmvk\" (UniqueName: \"kubernetes.io/projected/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-kube-api-access-ssmvk\") pod \"d70a5f1f-3650-4af3-ad6d-6ee96e50489e\" (UID: \"d70a5f1f-3650-4af3-ad6d-6ee96e50489e\") "
Oct 30 12:39:51.992563 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.992445    4945 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-util\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:51.992563 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.992464    4945 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-bundle\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:51.992563 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.992478    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6462\" (UniqueName: \"kubernetes.io/projected/edb1fef5-7b98-4348-8f6b-a0fca0dc14c2-kube-api-access-t6462\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:51.992745 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.992622    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-bundle" pod "d70a5f1f-3650-4af3-ad6d-6ee96e50489e" (UID: "d70a5f1f-3650-4af3-ad6d-6ee96e50489e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:39:51.993663 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.993541    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e66f60b4-ef15-4c45-bab3-873f4351e582-bundle" pod "e66f60b4-ef15-4c45-bab3-873f4351e582" (UID: "e66f60b4-ef15-4c45-bab3-873f4351e582"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:39:51.995192 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.995159    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-kube-api-access-ssmvk" pod "d70a5f1f-3650-4af3-ad6d-6ee96e50489e" (UID: "d70a5f1f-3650-4af3-ad6d-6ee96e50489e"). InnerVolumeSpecName "kube-api-access-ssmvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:39:51.995525 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:51.995493    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e66f60b4-ef15-4c45-bab3-873f4351e582-kube-api-access-tv6mh" pod "e66f60b4-ef15-4c45-bab3-873f4351e582" (UID: "e66f60b4-ef15-4c45-bab3-873f4351e582"). InnerVolumeSpecName "kube-api-access-tv6mh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:39:52.000648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.000529    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-util" pod "d70a5f1f-3650-4af3-ad6d-6ee96e50489e" (UID: "d70a5f1f-3650-4af3-ad6d-6ee96e50489e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:39:52.000648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.000571    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e66f60b4-ef15-4c45-bab3-873f4351e582-util" pod "e66f60b4-ef15-4c45-bab3-873f4351e582" (UID: "e66f60b4-ef15-4c45-bab3-873f4351e582"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:39:52.093648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.093570    4945 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-util\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:52.093648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.093614    4945 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e66f60b4-ef15-4c45-bab3-873f4351e582-util\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:52.093648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.093626    4945 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e66f60b4-ef15-4c45-bab3-873f4351e582-bundle\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:52.093648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.093634    4945 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-bundle\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:52.093648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.093642    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tv6mh\" (UniqueName: \"kubernetes.io/projected/e66f60b4-ef15-4c45-bab3-873f4351e582-kube-api-access-tv6mh\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:52.093648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.093651    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssmvk\" (UniqueName: \"kubernetes.io/projected/d70a5f1f-3650-4af3-ad6d-6ee96e50489e-kube-api-access-ssmvk\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:39:52.731283 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.731254    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x"
Oct 30 12:39:52.731283 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.731267    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644kjf7x" event={"ID":"e66f60b4-ef15-4c45-bab3-873f4351e582","Type":"ContainerDied","Data":"6103a706c9d5c1898ca82b8291add1d85390684a98e03abf2a6b011c40e52daa"}
Oct 30 12:39:52.731857 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.731325    4945 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6103a706c9d5c1898ca82b8291add1d85390684a98e03abf2a6b011c40e52daa"
Oct 30 12:39:52.733348 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.733325    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz"
Oct 30 12:39:52.733489 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.733338    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/fdb36248658848c45b2cd30265e4aa38f5747d5521cc1af4f2a8142cefht5mz" event={"ID":"edb1fef5-7b98-4348-8f6b-a0fca0dc14c2","Type":"ContainerDied","Data":"462ebac4fa67557848c045679b466f26368bb12fc69fb2f707370e4861fbb9f1"}
Oct 30 12:39:52.733489 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.733387    4945 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="462ebac4fa67557848c045679b466f26368bb12fc69fb2f707370e4861fbb9f1"
Oct 30 12:39:52.735921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.735890    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk" event={"ID":"d70a5f1f-3650-4af3-ad6d-6ee96e50489e","Type":"ContainerDied","Data":"fe3940400cbb37a0eff4634b11bb10c12acd31f1f8eab79d6132e6eb174b5b7c"}
Oct 30 12:39:52.736060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.735936    4945 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe3940400cbb37a0eff4634b11bb10c12acd31f1f8eab79d6132e6eb174b5b7c"
Oct 30 12:39:52.736060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:52.735991    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644gq7wk"
Oct 30 12:39:55.117673 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.117639    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7lldh"]
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.117929    4945 cpu_manager.go:414] "containerMap: removing container" podUID="e66f60b4-ef15-4c45-bab3-873f4351e582" containerName="util"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.117939    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e66f60b4-ef15-4c45-bab3-873f4351e582" containerName="util"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.117954    4945 cpu_manager.go:414] "containerMap: removing container" podUID="e66f60b4-ef15-4c45-bab3-873f4351e582" containerName="pull"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.117959    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e66f60b4-ef15-4c45-bab3-873f4351e582" containerName="pull"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.117966    4945 cpu_manager.go:414] "containerMap: removing container" podUID="d70a5f1f-3650-4af3-ad6d-6ee96e50489e" containerName="extract"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.117971    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d70a5f1f-3650-4af3-ad6d-6ee96e50489e" containerName="extract"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.117979    4945 cpu_manager.go:414] "containerMap: removing container" podUID="d3c33630-412a-4566-b890-04e0032d0e84" containerName="util"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.117984    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3c33630-412a-4566-b890-04e0032d0e84" containerName="util"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.117991    4945 cpu_manager.go:414] "containerMap: removing container" podUID="edb1fef5-7b98-4348-8f6b-a0fca0dc14c2" containerName="pull"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.117995    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="edb1fef5-7b98-4348-8f6b-a0fca0dc14c2" containerName="pull"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118002    4945 cpu_manager.go:414] "containerMap: removing container" podUID="edb1fef5-7b98-4348-8f6b-a0fca0dc14c2" containerName="util"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118006    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="edb1fef5-7b98-4348-8f6b-a0fca0dc14c2" containerName="util"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118012    4945 cpu_manager.go:414] "containerMap: removing container" podUID="d3c33630-412a-4566-b890-04e0032d0e84" containerName="extract"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118019    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3c33630-412a-4566-b890-04e0032d0e84" containerName="extract"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118024    4945 cpu_manager.go:414] "containerMap: removing container" podUID="edb1fef5-7b98-4348-8f6b-a0fca0dc14c2" containerName="extract"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118029    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="edb1fef5-7b98-4348-8f6b-a0fca0dc14c2" containerName="extract"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118037    4945 cpu_manager.go:414] "containerMap: removing container" podUID="add3deca-26ee-4b46-9877-68ec3c42fc73" containerName="util"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118051    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="add3deca-26ee-4b46-9877-68ec3c42fc73" containerName="util"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118058    4945 cpu_manager.go:414] "containerMap: removing container" podUID="add3deca-26ee-4b46-9877-68ec3c42fc73" containerName="pull"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118062    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="add3deca-26ee-4b46-9877-68ec3c42fc73" containerName="pull"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118068    4945 cpu_manager.go:414] "containerMap: removing container" podUID="d3c33630-412a-4566-b890-04e0032d0e84" containerName="pull"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118074    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3c33630-412a-4566-b890-04e0032d0e84" containerName="pull"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118080    4945 cpu_manager.go:414] "containerMap: removing container" podUID="89064220-965d-4d1e-be0d-69e90ddea2fd" containerName="extract"
Oct 30 12:39:55.118078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118085    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89064220-965d-4d1e-be0d-69e90ddea2fd" containerName="extract"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118093    4945 cpu_manager.go:414] "containerMap: removing container" podUID="add3deca-26ee-4b46-9877-68ec3c42fc73" containerName="extract"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118097    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="add3deca-26ee-4b46-9877-68ec3c42fc73" containerName="extract"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118110    4945 cpu_manager.go:414] "containerMap: removing container" podUID="d70a5f1f-3650-4af3-ad6d-6ee96e50489e" containerName="util"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118114    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d70a5f1f-3650-4af3-ad6d-6ee96e50489e" containerName="util"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118121    4945 cpu_manager.go:414] "containerMap: removing container" podUID="89064220-965d-4d1e-be0d-69e90ddea2fd" containerName="pull"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118125    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89064220-965d-4d1e-be0d-69e90ddea2fd" containerName="pull"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118130    4945 cpu_manager.go:414] "containerMap: removing container" podUID="d70a5f1f-3650-4af3-ad6d-6ee96e50489e" containerName="pull"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118134    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d70a5f1f-3650-4af3-ad6d-6ee96e50489e" containerName="pull"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118143    4945 cpu_manager.go:414] "containerMap: removing container" podUID="e66f60b4-ef15-4c45-bab3-873f4351e582" containerName="extract"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118147    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e66f60b4-ef15-4c45-bab3-873f4351e582" containerName="extract"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118154    4945 cpu_manager.go:414] "containerMap: removing container" podUID="89064220-965d-4d1e-be0d-69e90ddea2fd" containerName="util"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118158    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89064220-965d-4d1e-be0d-69e90ddea2fd" containerName="util"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118205    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="e66f60b4-ef15-4c45-bab3-873f4351e582" containerName="extract"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118213    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="d70a5f1f-3650-4af3-ad6d-6ee96e50489e" containerName="extract"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118219    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3c33630-412a-4566-b890-04e0032d0e84" containerName="extract"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118224    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="edb1fef5-7b98-4348-8f6b-a0fca0dc14c2" containerName="extract"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118230    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="89064220-965d-4d1e-be0d-69e90ddea2fd" containerName="extract"
Oct 30 12:39:55.118788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.118235    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="add3deca-26ee-4b46-9877-68ec3c42fc73" containerName="extract"
Oct 30 12:39:55.122247 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.122228    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7lldh"
Oct 30 12:39:55.136919 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.136894    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7lldh"]
Oct 30 12:39:55.146905 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.146883    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 30 12:39:55.147164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.147142    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-4249x\""
Oct 30 12:39:55.147240 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.147202    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 30 12:39:55.216829 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.216803    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41834164-ebad-4048-9863-7952fd3d8916-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7lldh\" (UID: \"41834164-ebad-4048-9863-7952fd3d8916\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7lldh"
Oct 30 12:39:55.216941 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.216922    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xv7q\" (UniqueName: \"kubernetes.io/projected/41834164-ebad-4048-9863-7952fd3d8916-kube-api-access-8xv7q\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7lldh\" (UID: \"41834164-ebad-4048-9863-7952fd3d8916\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7lldh"
Oct 30 12:39:55.318239 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.318208    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xv7q\" (UniqueName: \"kubernetes.io/projected/41834164-ebad-4048-9863-7952fd3d8916-kube-api-access-8xv7q\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7lldh\" (UID: \"41834164-ebad-4048-9863-7952fd3d8916\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7lldh"
Oct 30 12:39:55.318338 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.318247    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41834164-ebad-4048-9863-7952fd3d8916-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7lldh\" (UID: \"41834164-ebad-4048-9863-7952fd3d8916\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7lldh"
Oct 30 12:39:55.318719 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.318702    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41834164-ebad-4048-9863-7952fd3d8916-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7lldh\" (UID: \"41834164-ebad-4048-9863-7952fd3d8916\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7lldh"
Oct 30 12:39:55.337859 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.337837    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xv7q\" (UniqueName: \"kubernetes.io/projected/41834164-ebad-4048-9863-7952fd3d8916-kube-api-access-8xv7q\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-7lldh\" (UID: \"41834164-ebad-4048-9863-7952fd3d8916\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7lldh"
Oct 30 12:39:55.433238 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.433176    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7lldh"
Oct 30 12:39:55.751156 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:55.751126    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7lldh"]
Oct 30 12:39:55.754616 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:39:55.754575    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41834164_ebad_4048_9863_7952fd3d8916.slice/crio-97ce3b585ba7586b44a50d45c65dcab46713c2a99ec2394133eb0245e878b223 WatchSource:0}: Error finding container 97ce3b585ba7586b44a50d45c65dcab46713c2a99ec2394133eb0245e878b223: Status 404 returned error can't find the container with id 97ce3b585ba7586b44a50d45c65dcab46713c2a99ec2394133eb0245e878b223
Oct 30 12:39:56.190646 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:39:56.190546    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:39:56.754376 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:56.754334    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7lldh" event={"ID":"41834164-ebad-4048-9863-7952fd3d8916","Type":"ContainerStarted","Data":"97ce3b585ba7586b44a50d45c65dcab46713c2a99ec2394133eb0245e878b223"}
Oct 30 12:39:57.761030 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:57.760995    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7lldh" event={"ID":"41834164-ebad-4048-9863-7952fd3d8916","Type":"ContainerStarted","Data":"70a56ef8c1bd7c79098dd4031df580e1f0ffca5f29a4f074aa26c18a43691e6f"}
Oct 30 12:39:57.810361 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:57.810312    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-7lldh" podStartSLOduration=0.974440844 podStartE2EDuration="2.810299054s" totalImagesPullingTime="1.83585821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:39:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:39:57.808304132 +0000 UTC m=+242.386253223" watchObservedRunningTime="2026-10-30 12:39:57.810299054 +0000 UTC m=+242.388248136"
Oct 30 12:39:59.295488 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.295452    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"]
Oct 30 12:39:59.299123 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.299095    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.327998 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.327972    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 12:39:59.335279 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.335259    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 12:39:59.335382 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.335284    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 12:39:59.335382 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.335297    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-r2gbl\""
Oct 30 12:39:59.337812 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.337795    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 12:39:59.337889 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.337829    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 12:39:59.352706 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.352683    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/12634729-0110-49ae-993a-b8279e0acc40-apiservice-cert\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.352799 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.352713    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws6b9\" (UniqueName: \"kubernetes.io/projected/12634729-0110-49ae-993a-b8279e0acc40-kube-api-access-ws6b9\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.352799 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.352734    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/12634729-0110-49ae-993a-b8279e0acc40-metrics-cert\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.352799 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.352749    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/12634729-0110-49ae-993a-b8279e0acc40-manifests\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.352926 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.352891    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12634729-0110-49ae-993a-b8279e0acc40-webhook-cert\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.352926 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.352916    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12634729-0110-49ae-993a-b8279e0acc40-tmp\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.361251 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.361231    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"]
Oct 30 12:39:59.453498 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.453473    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws6b9\" (UniqueName: \"kubernetes.io/projected/12634729-0110-49ae-993a-b8279e0acc40-kube-api-access-ws6b9\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.453657 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.453510    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/12634729-0110-49ae-993a-b8279e0acc40-metrics-cert\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.453657 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.453529    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/12634729-0110-49ae-993a-b8279e0acc40-manifests\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.453776 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.453736    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12634729-0110-49ae-993a-b8279e0acc40-webhook-cert\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.453830 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.453783    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12634729-0110-49ae-993a-b8279e0acc40-tmp\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.453899 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.453886    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/12634729-0110-49ae-993a-b8279e0acc40-manifests\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.453967 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.453888    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/12634729-0110-49ae-993a-b8279e0acc40-apiservice-cert\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.455922 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.455900    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12634729-0110-49ae-993a-b8279e0acc40-tmp\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.456028 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.455931    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/12634729-0110-49ae-993a-b8279e0acc40-metrics-cert\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.456082 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.456024    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/12634729-0110-49ae-993a-b8279e0acc40-apiservice-cert\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.456346 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.456324    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/12634729-0110-49ae-993a-b8279e0acc40-webhook-cert\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.476197 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.476177    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws6b9\" (UniqueName: \"kubernetes.io/projected/12634729-0110-49ae-993a-b8279e0acc40-kube-api-access-ws6b9\") pod \"opendatahub-operator-controller-manager-786cc47469-ddlzq\" (UID: \"12634729-0110-49ae-993a-b8279e0acc40\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.614067 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.613985    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:39:59.975531 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:39:59.975498    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"]
Oct 30 12:39:59.978328 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:39:59.978297    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12634729_0110_49ae_993a_b8279e0acc40.slice/crio-e4a6b43ba57c69338482c106aba80c76e93bffbe7277119239f66187386ed6ad WatchSource:0}: Error finding container e4a6b43ba57c69338482c106aba80c76e93bffbe7277119239f66187386ed6ad: Status 404 returned error can't find the container with id e4a6b43ba57c69338482c106aba80c76e93bffbe7277119239f66187386ed6ad
Oct 30 12:40:00.777989 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:00.777946    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq" event={"ID":"12634729-0110-49ae-993a-b8279e0acc40","Type":"ContainerStarted","Data":"e4a6b43ba57c69338482c106aba80c76e93bffbe7277119239f66187386ed6ad"}
Oct 30 12:40:02.214138 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:02.214094    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-sdrvk"]
Oct 30 12:40:02.218291 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:02.218260    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdrvk"
Oct 30 12:40:02.231389 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:02.231364    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-sdrvk"]
Oct 30 12:40:02.240770 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:02.240741    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 12:40:02.240895 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:02.240741    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 12:40:02.240895 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:02.240839    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-q4lr6\""
Oct 30 12:40:02.276258 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:02.276230    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkcbk\" (UniqueName: \"kubernetes.io/projected/82c77b16-4b4f-49d4-bc77-63e24282324e-kube-api-access-dkcbk\") pod \"cert-manager-cainjector-5c57bbd4fc-sdrvk\" (UID: \"82c77b16-4b4f-49d4-bc77-63e24282324e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdrvk"
Oct 30 12:40:02.276416 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:02.276392    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82c77b16-4b4f-49d4-bc77-63e24282324e-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-sdrvk\" (UID: \"82c77b16-4b4f-49d4-bc77-63e24282324e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdrvk"
Oct 30 12:40:02.377549 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:02.377513    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82c77b16-4b4f-49d4-bc77-63e24282324e-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-sdrvk\" (UID: \"82c77b16-4b4f-49d4-bc77-63e24282324e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdrvk"
Oct 30 12:40:02.377739 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:02.377561    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkcbk\" (UniqueName: \"kubernetes.io/projected/82c77b16-4b4f-49d4-bc77-63e24282324e-kube-api-access-dkcbk\") pod \"cert-manager-cainjector-5c57bbd4fc-sdrvk\" (UID: \"82c77b16-4b4f-49d4-bc77-63e24282324e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdrvk"
Oct 30 12:40:02.399492 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:02.399464    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82c77b16-4b4f-49d4-bc77-63e24282324e-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-sdrvk\" (UID: \"82c77b16-4b4f-49d4-bc77-63e24282324e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdrvk"
Oct 30 12:40:02.399792 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:02.399770    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkcbk\" (UniqueName: \"kubernetes.io/projected/82c77b16-4b4f-49d4-bc77-63e24282324e-kube-api-access-dkcbk\") pod \"cert-manager-cainjector-5c57bbd4fc-sdrvk\" (UID: \"82c77b16-4b4f-49d4-bc77-63e24282324e\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdrvk"
Oct 30 12:40:02.543294 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:02.543216    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdrvk"
Oct 30 12:40:02.865331 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:02.865296    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-sdrvk"]
Oct 30 12:40:02.869090 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:40:02.869059    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82c77b16_4b4f_49d4_bc77_63e24282324e.slice/crio-07f989304c40edfca0da348fe482fbb8c6f9193e9b454f7a78c2ae3777f6316a WatchSource:0}: Error finding container 07f989304c40edfca0da348fe482fbb8c6f9193e9b454f7a78c2ae3777f6316a: Status 404 returned error can't find the container with id 07f989304c40edfca0da348fe482fbb8c6f9193e9b454f7a78c2ae3777f6316a
Oct 30 12:40:03.794143 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:03.794063    4945 generic.go:356] "Generic (PLEG): container finished" podID="12634729-0110-49ae-993a-b8279e0acc40" containerID="70da4c852bebc72dbda987cfe819e1aa870f6d3b2f32f59a104e8a675c6a4a2f" exitCode=0
Oct 30 12:40:03.794540 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:03.794149    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq" event={"ID":"12634729-0110-49ae-993a-b8279e0acc40","Type":"ContainerDied","Data":"70da4c852bebc72dbda987cfe819e1aa870f6d3b2f32f59a104e8a675c6a4a2f"}
Oct 30 12:40:03.795806 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:03.795497    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdrvk" event={"ID":"82c77b16-4b4f-49d4-bc77-63e24282324e","Type":"ContainerStarted","Data":"07f989304c40edfca0da348fe482fbb8c6f9193e9b454f7a78c2ae3777f6316a"}
Oct 30 12:40:04.802698 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:04.802658    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq" event={"ID":"12634729-0110-49ae-993a-b8279e0acc40","Type":"ContainerStarted","Data":"86698b0e252fd3aaff1fc6a0fb8c78ae8a15b1ae47aa38b80f8ca1df6388a426"}
Oct 30 12:40:04.803130 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:04.802852    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:40:04.900122 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:04.900063    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq" podStartSLOduration=2.1287204 podStartE2EDuration="5.900046746s" totalImagesPullingTime="3.226485573s" totalInitContainerRuntime="544.840773ms" isStatefulPod=false podCreationTimestamp="2026-10-30 12:39:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:40:04.887513157 +0000 UTC m=+249.465462245" watchObservedRunningTime="2026-10-30 12:40:04.900046746 +0000 UTC m=+249.477995837"
Oct 30 12:40:05.809388 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:05.809298    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdrvk" event={"ID":"82c77b16-4b4f-49d4-bc77-63e24282324e","Type":"ContainerStarted","Data":"49b7f28a417406bdbda78b1b36efa04cea9026e534cacc4ad0fb470306d4baad"}
Oct 30 12:40:05.886272 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:05.886217    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-sdrvk" podStartSLOduration=1.2827125129999999 podStartE2EDuration="3.886197812s" totalImagesPullingTime="2.603485299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:40:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:40:05.884229868 +0000 UTC m=+250.462178956" watchObservedRunningTime="2026-10-30 12:40:05.886197812 +0000 UTC m=+250.464146903"
Oct 30 12:40:11.741614 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:11.741574    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b575997cc-7g9t5"]
Oct 30 12:40:11.746778 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:11.746759    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b575997cc-7g9t5"
Oct 30 12:40:11.760557 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:11.760535    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b575997cc-7g9t5"]
Oct 30 12:40:11.770346 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:11.770302    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 12:40:11.770464 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:11.770387    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fg94b\""
Oct 30 12:40:11.770464 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:11.770422    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 12:40:11.858059 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:11.858021    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmq5q\" (UniqueName: \"kubernetes.io/projected/607ad48d-3968-4ffd-973f-51272eb00937-kube-api-access-zmq5q\") pod \"kuadrant-operator-controller-manager-6b575997cc-7g9t5\" (UID: \"607ad48d-3968-4ffd-973f-51272eb00937\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b575997cc-7g9t5"
Oct 30 12:40:11.959285 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:11.959253    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmq5q\" (UniqueName: \"kubernetes.io/projected/607ad48d-3968-4ffd-973f-51272eb00937-kube-api-access-zmq5q\") pod \"kuadrant-operator-controller-manager-6b575997cc-7g9t5\" (UID: \"607ad48d-3968-4ffd-973f-51272eb00937\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b575997cc-7g9t5"
Oct 30 12:40:11.982563 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:11.982534    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmq5q\" (UniqueName: \"kubernetes.io/projected/607ad48d-3968-4ffd-973f-51272eb00937-kube-api-access-zmq5q\") pod \"kuadrant-operator-controller-manager-6b575997cc-7g9t5\" (UID: \"607ad48d-3968-4ffd-973f-51272eb00937\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b575997cc-7g9t5"
Oct 30 12:40:12.059316 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:12.059259    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b575997cc-7g9t5"
Oct 30 12:40:12.401271 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:12.401172    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b575997cc-7g9t5"]
Oct 30 12:40:12.416142 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:40:12.416110    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod607ad48d_3968_4ffd_973f_51272eb00937.slice/crio-c54e988ff06635d76c633f3a02421e8e74dd1906ed0d84d5d5e07210761bd3a0 WatchSource:0}: Error finding container c54e988ff06635d76c633f3a02421e8e74dd1906ed0d84d5d5e07210761bd3a0: Status 404 returned error can't find the container with id c54e988ff06635d76c633f3a02421e8e74dd1906ed0d84d5d5e07210761bd3a0
Oct 30 12:40:12.841592 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:12.841559    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b575997cc-7g9t5" event={"ID":"607ad48d-3968-4ffd-973f-51272eb00937","Type":"ContainerStarted","Data":"c54e988ff06635d76c633f3a02421e8e74dd1906ed0d84d5d5e07210761bd3a0"}
Oct 30 12:40:15.812488 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:15.812459    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-786cc47469-ddlzq"
Oct 30 12:40:17.870890 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:17.870850    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b575997cc-7g9t5" event={"ID":"607ad48d-3968-4ffd-973f-51272eb00937","Type":"ContainerStarted","Data":"7f1753e3d8b8e2f4971fe5691361adfaa37bc4a0a494572117b9cf26e7d4cec2"}
Oct 30 12:40:17.871292 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:17.870935    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b575997cc-7g9t5"
Oct 30 12:40:17.943163 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:17.943102    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b575997cc-7g9t5" podStartSLOduration=2.12294619 podStartE2EDuration="6.943088513s" totalImagesPullingTime="4.820142323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:40:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:40:17.930325118 +0000 UTC m=+262.508274206" watchObservedRunningTime="2026-10-30 12:40:17.943088513 +0000 UTC m=+262.521037603"
Oct 30 12:40:23.551189 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.551147    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-57f5c66455-9g9vc"]
Oct 30 12:40:23.553730 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.553713    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-57f5c66455-9g9vc"
Oct 30 12:40:23.579959 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.579938    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 30 12:40:23.591680 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.591656    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-66cwx\""
Oct 30 12:40:23.594394 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.594371    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-57f5c66455-9g9vc"]
Oct 30 12:40:23.659528 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.659501    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s48v7\" (UniqueName: \"kubernetes.io/projected/e0821ac0-2e9b-4403-914a-e13b7130e8a2-kube-api-access-s48v7\") pod \"dns-operator-controller-manager-57f5c66455-9g9vc\" (UID: \"e0821ac0-2e9b-4403-914a-e13b7130e8a2\") " pod="kuadrant-system/dns-operator-controller-manager-57f5c66455-9g9vc"
Oct 30 12:40:23.758771 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.758733    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/mcp-gateway-controller-6c98b995c5-j98z2"]
Oct 30 12:40:23.760815 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.760792    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s48v7\" (UniqueName: \"kubernetes.io/projected/e0821ac0-2e9b-4403-914a-e13b7130e8a2-kube-api-access-s48v7\") pod \"dns-operator-controller-manager-57f5c66455-9g9vc\" (UID: \"e0821ac0-2e9b-4403-914a-e13b7130e8a2\") " pod="kuadrant-system/dns-operator-controller-manager-57f5c66455-9g9vc"
Oct 30 12:40:23.762082 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.762065    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/mcp-gateway-controller-6c98b995c5-j98z2"
Oct 30 12:40:23.778794 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.778774    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"mcp-gateway-controller-dockercfg-ml4n4\""
Oct 30 12:40:23.785698 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.785671    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/mcp-gateway-controller-6c98b995c5-j98z2"]
Oct 30 12:40:23.786578 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.786556    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s48v7\" (UniqueName: \"kubernetes.io/projected/e0821ac0-2e9b-4403-914a-e13b7130e8a2-kube-api-access-s48v7\") pod \"dns-operator-controller-manager-57f5c66455-9g9vc\" (UID: \"e0821ac0-2e9b-4403-914a-e13b7130e8a2\") " pod="kuadrant-system/dns-operator-controller-manager-57f5c66455-9g9vc"
Oct 30 12:40:23.861276 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.861214    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sng4d\" (UniqueName: \"kubernetes.io/projected/a59f98de-82e6-4a91-bdab-beb39922f5ff-kube-api-access-sng4d\") pod \"mcp-gateway-controller-6c98b995c5-j98z2\" (UID: \"a59f98de-82e6-4a91-bdab-beb39922f5ff\") " pod="kuadrant-system/mcp-gateway-controller-6c98b995c5-j98z2"
Oct 30 12:40:23.866157 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.866129    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-57f5c66455-9g9vc"
Oct 30 12:40:23.962289 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.962257    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sng4d\" (UniqueName: \"kubernetes.io/projected/a59f98de-82e6-4a91-bdab-beb39922f5ff-kube-api-access-sng4d\") pod \"mcp-gateway-controller-6c98b995c5-j98z2\" (UID: \"a59f98de-82e6-4a91-bdab-beb39922f5ff\") " pod="kuadrant-system/mcp-gateway-controller-6c98b995c5-j98z2"
Oct 30 12:40:23.981884 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:23.981860    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sng4d\" (UniqueName: \"kubernetes.io/projected/a59f98de-82e6-4a91-bdab-beb39922f5ff-kube-api-access-sng4d\") pod \"mcp-gateway-controller-6c98b995c5-j98z2\" (UID: \"a59f98de-82e6-4a91-bdab-beb39922f5ff\") " pod="kuadrant-system/mcp-gateway-controller-6c98b995c5-j98z2"
Oct 30 12:40:24.075245 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:24.075218    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/mcp-gateway-controller-6c98b995c5-j98z2"
Oct 30 12:40:24.160245 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:24.160211    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-57f5c66455-9g9vc"]
Oct 30 12:40:24.164690 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:40:24.164662    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0821ac0_2e9b_4403_914a_e13b7130e8a2.slice/crio-1a636b4602f969bac0b1f67914eb1a1890f2f7be7a2d0b7eb369732350cfeea7 WatchSource:0}: Error finding container 1a636b4602f969bac0b1f67914eb1a1890f2f7be7a2d0b7eb369732350cfeea7: Status 404 returned error can't find the container with id 1a636b4602f969bac0b1f67914eb1a1890f2f7be7a2d0b7eb369732350cfeea7
Oct 30 12:40:24.392802 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:24.392724    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-779bff956b-l69xd"]
Oct 30 12:40:24.397485 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:24.397456    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-779bff956b-l69xd"
Oct 30 12:40:24.403325 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:24.403297    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/mcp-gateway-controller-6c98b995c5-j98z2"]
Oct 30 12:40:24.407472 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:40:24.407443    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda59f98de_82e6_4a91_bdab_beb39922f5ff.slice/crio-3960ee53b13cb622a9e56515a4ba324bf5fd069599389330e677176b47135cc7 WatchSource:0}: Error finding container 3960ee53b13cb622a9e56515a4ba324bf5fd069599389330e677176b47135cc7: Status 404 returned error can't find the container with id 3960ee53b13cb622a9e56515a4ba324bf5fd069599389330e677176b47135cc7
Oct 30 12:40:24.430835 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:24.430809    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-779bff956b-l69xd"]
Oct 30 12:40:24.432786 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:24.432767    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-h6lhm\""
Oct 30 12:40:24.467463 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:24.467441    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jsrp\" (UniqueName: \"kubernetes.io/projected/f6d044ab-287d-434c-8fd5-bb9b61eaffaa-kube-api-access-4jsrp\") pod \"authorino-operator-779bff956b-l69xd\" (UID: \"f6d044ab-287d-434c-8fd5-bb9b61eaffaa\") " pod="kuadrant-system/authorino-operator-779bff956b-l69xd"
Oct 30 12:40:24.568045 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:24.568008    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jsrp\" (UniqueName: \"kubernetes.io/projected/f6d044ab-287d-434c-8fd5-bb9b61eaffaa-kube-api-access-4jsrp\") pod \"authorino-operator-779bff956b-l69xd\" (UID: \"f6d044ab-287d-434c-8fd5-bb9b61eaffaa\") " pod="kuadrant-system/authorino-operator-779bff956b-l69xd"
Oct 30 12:40:24.586456 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:24.586428    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jsrp\" (UniqueName: \"kubernetes.io/projected/f6d044ab-287d-434c-8fd5-bb9b61eaffaa-kube-api-access-4jsrp\") pod \"authorino-operator-779bff956b-l69xd\" (UID: \"f6d044ab-287d-434c-8fd5-bb9b61eaffaa\") " pod="kuadrant-system/authorino-operator-779bff956b-l69xd"
Oct 30 12:40:24.712627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:24.712566    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-779bff956b-l69xd"
Oct 30 12:40:24.905369 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:24.905332    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/mcp-gateway-controller-6c98b995c5-j98z2" event={"ID":"a59f98de-82e6-4a91-bdab-beb39922f5ff","Type":"ContainerStarted","Data":"3960ee53b13cb622a9e56515a4ba324bf5fd069599389330e677176b47135cc7"}
Oct 30 12:40:24.906579 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:24.906554    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-57f5c66455-9g9vc" event={"ID":"e0821ac0-2e9b-4403-914a-e13b7130e8a2","Type":"ContainerStarted","Data":"1a636b4602f969bac0b1f67914eb1a1890f2f7be7a2d0b7eb369732350cfeea7"}
Oct 30 12:40:25.006196 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:25.006127    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-779bff956b-l69xd"]
Oct 30 12:40:25.009274 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:40:25.009249    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d044ab_287d_434c_8fd5_bb9b61eaffaa.slice/crio-5f6b3efb69b92dcf70054c31a5f69d58838c88325f376cbe4c59f89c6df6a893 WatchSource:0}: Error finding container 5f6b3efb69b92dcf70054c31a5f69d58838c88325f376cbe4c59f89c6df6a893: Status 404 returned error can't find the container with id 5f6b3efb69b92dcf70054c31a5f69d58838c88325f376cbe4c59f89c6df6a893
Oct 30 12:40:25.196451 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:25.196422    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/developer-portal-controller-66d8dbb8c8-jxp7q"]
Oct 30 12:40:25.200955 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:25.200938    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/developer-portal-controller-66d8dbb8c8-jxp7q"
Oct 30 12:40:25.218271 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:25.218250    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"developer-portal-controller-manager-dockercfg-5hw6v\""
Oct 30 12:40:25.219549 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:25.219527    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/developer-portal-controller-66d8dbb8c8-jxp7q"]
Oct 30 12:40:25.274200 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:25.274134    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcp45\" (UniqueName: \"kubernetes.io/projected/2b240210-5d58-4791-8745-60c58f3e9bbf-kube-api-access-zcp45\") pod \"developer-portal-controller-66d8dbb8c8-jxp7q\" (UID: \"2b240210-5d58-4791-8745-60c58f3e9bbf\") " pod="kuadrant-system/developer-portal-controller-66d8dbb8c8-jxp7q"
Oct 30 12:40:25.375334 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:25.375301    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcp45\" (UniqueName: \"kubernetes.io/projected/2b240210-5d58-4791-8745-60c58f3e9bbf-kube-api-access-zcp45\") pod \"developer-portal-controller-66d8dbb8c8-jxp7q\" (UID: \"2b240210-5d58-4791-8745-60c58f3e9bbf\") " pod="kuadrant-system/developer-portal-controller-66d8dbb8c8-jxp7q"
Oct 30 12:40:25.398942 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:25.398913    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcp45\" (UniqueName: \"kubernetes.io/projected/2b240210-5d58-4791-8745-60c58f3e9bbf-kube-api-access-zcp45\") pod \"developer-portal-controller-66d8dbb8c8-jxp7q\" (UID: \"2b240210-5d58-4791-8745-60c58f3e9bbf\") " pod="kuadrant-system/developer-portal-controller-66d8dbb8c8-jxp7q"
Oct 30 12:40:25.513867 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:25.513832    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/developer-portal-controller-66d8dbb8c8-jxp7q"
Oct 30 12:40:25.815347 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:25.815213    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/developer-portal-controller-66d8dbb8c8-jxp7q"]
Oct 30 12:40:25.826198 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:40:25.826160    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b240210_5d58_4791_8745_60c58f3e9bbf.slice/crio-6c05d5589d10fbdf6ae9c6cd493f0e918aa7ec8dbfa10ec380b939060d06d89b WatchSource:0}: Error finding container 6c05d5589d10fbdf6ae9c6cd493f0e918aa7ec8dbfa10ec380b939060d06d89b: Status 404 returned error can't find the container with id 6c05d5589d10fbdf6ae9c6cd493f0e918aa7ec8dbfa10ec380b939060d06d89b
Oct 30 12:40:25.915662 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:25.915613    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-779bff956b-l69xd" event={"ID":"f6d044ab-287d-434c-8fd5-bb9b61eaffaa","Type":"ContainerStarted","Data":"5f6b3efb69b92dcf70054c31a5f69d58838c88325f376cbe4c59f89c6df6a893"}
Oct 30 12:40:25.917587 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:25.917555    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/developer-portal-controller-66d8dbb8c8-jxp7q" event={"ID":"2b240210-5d58-4791-8745-60c58f3e9bbf","Type":"ContainerStarted","Data":"6c05d5589d10fbdf6ae9c6cd493f0e918aa7ec8dbfa10ec380b939060d06d89b"}
Oct 30 12:40:26.199197 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:40:26.199157    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:40:26.927462 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:26.926586    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/mcp-gateway-controller-6c98b995c5-j98z2" event={"ID":"a59f98de-82e6-4a91-bdab-beb39922f5ff","Type":"ContainerStarted","Data":"07afdcf3bc2011313abb711f90df47dd427fdf810a1652ba89c7deb3d8b21515"}
Oct 30 12:40:26.927462 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:26.927409    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/mcp-gateway-controller-6c98b995c5-j98z2"
Oct 30 12:40:26.972416 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:26.972335    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/mcp-gateway-controller-6c98b995c5-j98z2" podStartSLOduration=2.043714456 podStartE2EDuration="3.972323475s" totalImagesPullingTime="1.928609019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:40:26.970873152 +0000 UTC m=+271.548822246" watchObservedRunningTime="2026-10-30 12:40:26.972323475 +0000 UTC m=+271.550272565"
Oct 30 12:40:28.877385 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:28.877352    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b575997cc-7g9t5"
Oct 30 12:40:29.943381 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:29.943340    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-779bff956b-l69xd" event={"ID":"f6d044ab-287d-434c-8fd5-bb9b61eaffaa","Type":"ContainerStarted","Data":"e707aaf67f9ab3b85e57e7219dbf8a827ddb8d44fcfd1bb9fd23fbb2744b0ccb"}
Oct 30 12:40:29.943846 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:29.943494    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-779bff956b-l69xd"
Oct 30 12:40:29.945044 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:29.945004    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/developer-portal-controller-66d8dbb8c8-jxp7q" event={"ID":"2b240210-5d58-4791-8745-60c58f3e9bbf","Type":"ContainerStarted","Data":"da06f1ac7e543a7a6ba0a5a500fb3830d144d621aa70c1c2355ac821fe64111d"}
Oct 30 12:40:29.945184 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:29.945141    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/developer-portal-controller-66d8dbb8c8-jxp7q"
Oct 30 12:40:29.946467 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:29.946448    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-57f5c66455-9g9vc" event={"ID":"e0821ac0-2e9b-4403-914a-e13b7130e8a2","Type":"ContainerStarted","Data":"8d64730d28df7e27ac9e4d4a615ea5870d38391a21d4242d6f3355e1866914ab"}
Oct 30 12:40:29.946547 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:29.946488    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-57f5c66455-9g9vc"
Oct 30 12:40:29.979556 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:29.979513    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-779bff956b-l69xd" podStartSLOduration=2.066771898 podStartE2EDuration="5.979501685s" totalImagesPullingTime="3.912729787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:40:29.97934052 +0000 UTC m=+274.557289609" watchObservedRunningTime="2026-10-30 12:40:29.979501685 +0000 UTC m=+274.557450773"
Oct 30 12:40:30.017280 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:30.017237    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-57f5c66455-9g9vc" podStartSLOduration=2.262457773 podStartE2EDuration="7.017224941s" totalImagesPullingTime="4.754767168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:40:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:40:30.015358071 +0000 UTC m=+274.593307160" watchObservedRunningTime="2026-10-30 12:40:30.017224941 +0000 UTC m=+274.595174030"
Oct 30 12:40:30.053571 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:30.053529    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/developer-portal-controller-66d8dbb8c8-jxp7q" podStartSLOduration=1.9041839440000001 podStartE2EDuration="5.053516425s" totalImagesPullingTime="3.149332481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:40:30.052337037 +0000 UTC m=+274.630286126" watchObservedRunningTime="2026-10-30 12:40:30.053516425 +0000 UTC m=+274.631465514"
Oct 30 12:40:38.939736 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:38.939703    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/mcp-gateway-controller-6c98b995c5-j98z2"
Oct 30 12:40:40.954104 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:40.954074    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/developer-portal-controller-66d8dbb8c8-jxp7q"
Oct 30 12:40:40.954640 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:40.954152    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-779bff956b-l69xd"
Oct 30 12:40:40.954640 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:40.954186    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-57f5c66455-9g9vc"
Oct 30 12:40:54.613075 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.613039    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"]
Oct 30 12:40:54.621142 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.621113    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.638112 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.638080    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"]
Oct 30 12:40:54.685413 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.685379    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7vk2z\""
Oct 30 12:40:54.708904 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.708868    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1c8ceec-1c52-4523-ae41-69be10ac654d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.709106 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.708916    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1c8ceec-1c52-4523-ae41-69be10ac654d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.709106 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.708968    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1c8ceec-1c52-4523-ae41-69be10ac654d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.709106 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.708988    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1c8ceec-1c52-4523-ae41-69be10ac654d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.709106 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.709008    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1c8ceec-1c52-4523-ae41-69be10ac654d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.709106 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.709029    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmddn\" (UniqueName: \"kubernetes.io/projected/b1c8ceec-1c52-4523-ae41-69be10ac654d-kube-api-access-rmddn\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.709321 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.709161    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1c8ceec-1c52-4523-ae41-69be10ac654d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.709321 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.709193    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1c8ceec-1c52-4523-ae41-69be10ac654d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.709321 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.709214    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1c8ceec-1c52-4523-ae41-69be10ac654d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.734057 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.734027    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-77dd99c685-ffnbm"]
Oct 30 12:40:54.737347 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.737328    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm"
Oct 30 12:40:54.757369 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.757346    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5twgk\""
Oct 30 12:40:54.757796 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.757771    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 12:40:54.759511 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.759488    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-77dd99c685-ffnbm"]
Oct 30 12:40:54.810274 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.810242    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1c8ceec-1c52-4523-ae41-69be10ac654d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.810449 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.810288    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1c8ceec-1c52-4523-ae41-69be10ac654d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.810449 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.810324    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/32d480e2-e304-4124-adf0-8cc0e31c9acf-config-file\") pod \"limitador-limitador-77dd99c685-ffnbm\" (UID: \"32d480e2-e304-4124-adf0-8cc0e31c9acf\") " pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm"
Oct 30 12:40:54.810449 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.810357    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1c8ceec-1c52-4523-ae41-69be10ac654d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.810449 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.810396    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1c8ceec-1c52-4523-ae41-69be10ac654d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.810709 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.810467    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1c8ceec-1c52-4523-ae41-69be10ac654d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.810709 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.810524    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1c8ceec-1c52-4523-ae41-69be10ac654d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.810709 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.810575    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1c8ceec-1c52-4523-ae41-69be10ac654d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.810709 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.810633    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmddn\" (UniqueName: \"kubernetes.io/projected/b1c8ceec-1c52-4523-ae41-69be10ac654d-kube-api-access-rmddn\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.810914 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.810728    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1c8ceec-1c52-4523-ae41-69be10ac654d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.810914 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.810740    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1c8ceec-1c52-4523-ae41-69be10ac654d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.810914 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.810758    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9dsr\" (UniqueName: \"kubernetes.io/projected/32d480e2-e304-4124-adf0-8cc0e31c9acf-kube-api-access-c9dsr\") pod \"limitador-limitador-77dd99c685-ffnbm\" (UID: \"32d480e2-e304-4124-adf0-8cc0e31c9acf\") " pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm"
Oct 30 12:40:54.810914 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.810796    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1c8ceec-1c52-4523-ae41-69be10ac654d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.810914 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.810835    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1c8ceec-1c52-4523-ae41-69be10ac654d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.811170 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.811098    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1c8ceec-1c52-4523-ae41-69be10ac654d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.811786 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.811755    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1c8ceec-1c52-4523-ae41-69be10ac654d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.813014 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.812984    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1c8ceec-1c52-4523-ae41-69be10ac654d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.813276 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.813257    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1c8ceec-1c52-4523-ae41-69be10ac654d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.830526 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.830497    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1c8ceec-1c52-4523-ae41-69be10ac654d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.830734 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.830703    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmddn\" (UniqueName: \"kubernetes.io/projected/b1c8ceec-1c52-4523-ae41-69be10ac654d-kube-api-access-rmddn\") pod \"data-science-gateway-data-science-gateway-class-858f55f4d9t74ft\" (UID: \"b1c8ceec-1c52-4523-ae41-69be10ac654d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:54.866306 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.866237    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-77dd99c685-ffnbm"]
Oct 30 12:40:54.911412 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.911380    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9dsr\" (UniqueName: \"kubernetes.io/projected/32d480e2-e304-4124-adf0-8cc0e31c9acf-kube-api-access-c9dsr\") pod \"limitador-limitador-77dd99c685-ffnbm\" (UID: \"32d480e2-e304-4124-adf0-8cc0e31c9acf\") " pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm"
Oct 30 12:40:54.911583 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.911423    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/32d480e2-e304-4124-adf0-8cc0e31c9acf-config-file\") pod \"limitador-limitador-77dd99c685-ffnbm\" (UID: \"32d480e2-e304-4124-adf0-8cc0e31c9acf\") " pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm"
Oct 30 12:40:54.912086 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.912060    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/32d480e2-e304-4124-adf0-8cc0e31c9acf-config-file\") pod \"limitador-limitador-77dd99c685-ffnbm\" (UID: \"32d480e2-e304-4124-adf0-8cc0e31c9acf\") " pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm"
Oct 30 12:40:54.930050 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.930013    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9dsr\" (UniqueName: \"kubernetes.io/projected/32d480e2-e304-4124-adf0-8cc0e31c9acf-kube-api-access-c9dsr\") pod \"limitador-limitador-77dd99c685-ffnbm\" (UID: \"32d480e2-e304-4124-adf0-8cc0e31c9acf\") " pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm"
Oct 30 12:40:54.933750 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:54.933725    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:55.049138 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:55.049103    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm"
Oct 30 12:40:55.248171 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:55.247769    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"]
Oct 30 12:40:55.254068 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:40:55.254038    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1c8ceec_1c52_4523_ae41_69be10ac654d.slice/crio-20dd885688e0a54d8632f35091b0721536bf9afd641de6422c5a92769cbd0518 WatchSource:0}: Error finding container 20dd885688e0a54d8632f35091b0721536bf9afd641de6422c5a92769cbd0518: Status 404 returned error can't find the container with id 20dd885688e0a54d8632f35091b0721536bf9afd641de6422c5a92769cbd0518
Oct 30 12:40:55.256828 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:55.256796    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:40:55.256942 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:55.256879    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:40:55.256942 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:55.256925    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:40:55.347266 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:55.347235    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-77dd99c685-ffnbm"]
Oct 30 12:40:55.351856 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:40:55.351824    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32d480e2_e304_4124_adf0_8cc0e31c9acf.slice/crio-a79f1fb8104aede59320cdec65f51f51fa9d7804c57f9dc9784060c9c2a78610 WatchSource:0}: Error finding container a79f1fb8104aede59320cdec65f51f51fa9d7804c57f9dc9784060c9c2a78610: Status 404 returned error can't find the container with id a79f1fb8104aede59320cdec65f51f51fa9d7804c57f9dc9784060c9c2a78610
Oct 30 12:40:55.580577 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:55.580499    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 12:40:55.580943 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:55.580919    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 12:40:56.080843 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:56.080803    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft" event={"ID":"b1c8ceec-1c52-4523-ae41-69be10ac654d","Type":"ContainerStarted","Data":"3d5421205cff86ff9a128ec583a98f51bcf78d8ea7a49886df399297d581884d"}
Oct 30 12:40:56.081269 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:56.080871    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft" event={"ID":"b1c8ceec-1c52-4523-ae41-69be10ac654d","Type":"ContainerStarted","Data":"20dd885688e0a54d8632f35091b0721536bf9afd641de6422c5a92769cbd0518"}
Oct 30 12:40:56.082038 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:56.082013    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm" event={"ID":"32d480e2-e304-4124-adf0-8cc0e31c9acf","Type":"ContainerStarted","Data":"a79f1fb8104aede59320cdec65f51f51fa9d7804c57f9dc9784060c9c2a78610"}
Oct 30 12:40:56.135235 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:56.135182    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft" podStartSLOduration=2.135167164 podStartE2EDuration="2.135167164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:40:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:40:56.133035984 +0000 UTC m=+300.710985083" watchObservedRunningTime="2026-10-30 12:40:56.135167164 +0000 UTC m=+300.713116252"
Oct 30 12:40:56.194669 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:40:56.194628    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:40:56.934223 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:56.934133    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:56.939228 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:56.939205    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:57.088038 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:57.088004    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm" event={"ID":"32d480e2-e304-4124-adf0-8cc0e31c9acf","Type":"ContainerStarted","Data":"60268e39f39b8e02c74ae9839b7f367f15cc95db3dd119c221ec9bae64f1f81f"}
Oct 30 12:40:57.088520 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:57.088222    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:57.088520 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:57.088263    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm"
Oct 30 12:40:57.089161 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:57.089140    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9t74ft"
Oct 30 12:40:57.125422 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:57.125379    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm" podStartSLOduration=1.873750139 podStartE2EDuration="3.12536441s" totalImagesPullingTime="1.251614271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:40:57.123648073 +0000 UTC m=+301.701597168" watchObservedRunningTime="2026-10-30 12:40:57.12536441 +0000 UTC m=+301.703313499"
Oct 30 12:40:58.333182 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.333145    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"]
Oct 30 12:40:58.339800 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.339777    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:40:58.367219 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.367195    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 12:40:58.367345 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.367216    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-jsdqg\""
Oct 30 12:40:58.367405 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.367391    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 12:40:58.391149 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.391107    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 30 12:40:58.393274 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.393255    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"]
Oct 30 12:40:58.448480 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.448451    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zmbk\" (UniqueName: \"kubernetes.io/projected/85cd2a7e-fce3-43f2-8d41-feaf9406baa5-kube-api-access-9zmbk\") pod \"kserve-module-controller-manager-7566cf5fc8-s9qks\" (UID: \"85cd2a7e-fce3-43f2-8d41-feaf9406baa5\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:40:58.448594 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.448488    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85cd2a7e-fce3-43f2-8d41-feaf9406baa5-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-s9qks\" (UID: \"85cd2a7e-fce3-43f2-8d41-feaf9406baa5\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:40:58.448594 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.448516    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/85cd2a7e-fce3-43f2-8d41-feaf9406baa5-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-s9qks\" (UID: \"85cd2a7e-fce3-43f2-8d41-feaf9406baa5\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:40:58.448687 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.448627    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85cd2a7e-fce3-43f2-8d41-feaf9406baa5-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-s9qks\" (UID: \"85cd2a7e-fce3-43f2-8d41-feaf9406baa5\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:40:58.549590 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.549553    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zmbk\" (UniqueName: \"kubernetes.io/projected/85cd2a7e-fce3-43f2-8d41-feaf9406baa5-kube-api-access-9zmbk\") pod \"kserve-module-controller-manager-7566cf5fc8-s9qks\" (UID: \"85cd2a7e-fce3-43f2-8d41-feaf9406baa5\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:40:58.549590 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.549591    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85cd2a7e-fce3-43f2-8d41-feaf9406baa5-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-s9qks\" (UID: \"85cd2a7e-fce3-43f2-8d41-feaf9406baa5\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:40:58.549831 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.549651    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/85cd2a7e-fce3-43f2-8d41-feaf9406baa5-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-s9qks\" (UID: \"85cd2a7e-fce3-43f2-8d41-feaf9406baa5\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:40:58.549831 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.549682    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85cd2a7e-fce3-43f2-8d41-feaf9406baa5-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-s9qks\" (UID: \"85cd2a7e-fce3-43f2-8d41-feaf9406baa5\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:40:58.550141 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.550106    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85cd2a7e-fce3-43f2-8d41-feaf9406baa5-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-s9qks\" (UID: \"85cd2a7e-fce3-43f2-8d41-feaf9406baa5\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:40:58.550206 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.550152    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/85cd2a7e-fce3-43f2-8d41-feaf9406baa5-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-s9qks\" (UID: \"85cd2a7e-fce3-43f2-8d41-feaf9406baa5\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:40:58.550570 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.550550    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/85cd2a7e-fce3-43f2-8d41-feaf9406baa5-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-s9qks\" (UID: \"85cd2a7e-fce3-43f2-8d41-feaf9406baa5\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:40:58.568855 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.568832    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zmbk\" (UniqueName: \"kubernetes.io/projected/85cd2a7e-fce3-43f2-8d41-feaf9406baa5-kube-api-access-9zmbk\") pod \"kserve-module-controller-manager-7566cf5fc8-s9qks\" (UID: \"85cd2a7e-fce3-43f2-8d41-feaf9406baa5\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:40:58.652522 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.652453    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:40:58.957129 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:58.957097    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"]
Oct 30 12:40:58.961984 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:40:58.961959    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85cd2a7e_fce3_43f2_8d41_feaf9406baa5.slice/crio-2987da84bd7befe4ab7710d65aeecfbbd70c18c59a4b93ecd6459d73a7eaceaa WatchSource:0}: Error finding container 2987da84bd7befe4ab7710d65aeecfbbd70c18c59a4b93ecd6459d73a7eaceaa: Status 404 returned error can't find the container with id 2987da84bd7befe4ab7710d65aeecfbbd70c18c59a4b93ecd6459d73a7eaceaa
Oct 30 12:40:59.099880 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:40:59.099837    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks" event={"ID":"85cd2a7e-fce3-43f2-8d41-feaf9406baa5","Type":"ContainerStarted","Data":"2987da84bd7befe4ab7710d65aeecfbbd70c18c59a4b93ecd6459d73a7eaceaa"}
Oct 30 12:41:02.118337 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:02.118290    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks" event={"ID":"85cd2a7e-fce3-43f2-8d41-feaf9406baa5","Type":"ContainerStarted","Data":"c5b4543d3b8f7f145a1faa256fb92fd58fa6de9a138766a1fceaa8b522ac597a"}
Oct 30 12:41:02.118805 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:02.118444    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:41:02.169666 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:02.169621    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks" podStartSLOduration=1.9273393269999999 podStartE2EDuration="4.169608729s" totalImagesPullingTime="2.242269402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:41:02.168319537 +0000 UTC m=+306.746268625" watchObservedRunningTime="2026-10-30 12:41:02.169608729 +0000 UTC m=+306.747557808"
Oct 30 12:41:08.094385 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:08.094305    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm"
Oct 30 12:41:11.556038 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:11.556004    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-77dd99c685-ffnbm"]
Oct 30 12:41:11.556474 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:11.556185    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm" podUID="32d480e2-e304-4124-adf0-8cc0e31c9acf" containerName="limitador" containerID="cri-o://60268e39f39b8e02c74ae9839b7f367f15cc95db3dd119c221ec9bae64f1f81f" gracePeriod=30
Oct 30 12:41:12.104146 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.104117    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm"
Oct 30 12:41:12.169750 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.169664    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32d480e2-e304-4124-adf0-8cc0e31c9acf-config-file\" (UniqueName: \"kubernetes.io/configmap/32d480e2-e304-4124-adf0-8cc0e31c9acf-config-file\") pod \"32d480e2-e304-4124-adf0-8cc0e31c9acf\" (UID: \"32d480e2-e304-4124-adf0-8cc0e31c9acf\") "
Oct 30 12:41:12.169911 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.169754    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32d480e2-e304-4124-adf0-8cc0e31c9acf-kube-api-access-c9dsr\" (UniqueName: \"kubernetes.io/projected/32d480e2-e304-4124-adf0-8cc0e31c9acf-kube-api-access-c9dsr\") pod \"32d480e2-e304-4124-adf0-8cc0e31c9acf\" (UID: \"32d480e2-e304-4124-adf0-8cc0e31c9acf\") "
Oct 30 12:41:12.170150 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.170113    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32d480e2-e304-4124-adf0-8cc0e31c9acf-config-file" pod "32d480e2-e304-4124-adf0-8cc0e31c9acf" (UID: "32d480e2-e304-4124-adf0-8cc0e31c9acf"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:41:12.171101 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.171079    4945 generic.go:356] "Generic (PLEG): container finished" podID="32d480e2-e304-4124-adf0-8cc0e31c9acf" containerID="60268e39f39b8e02c74ae9839b7f367f15cc95db3dd119c221ec9bae64f1f81f" exitCode=0
Oct 30 12:41:12.171197 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.171141    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm"
Oct 30 12:41:12.171197 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.171163    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm" event={"ID":"32d480e2-e304-4124-adf0-8cc0e31c9acf","Type":"ContainerDied","Data":"60268e39f39b8e02c74ae9839b7f367f15cc95db3dd119c221ec9bae64f1f81f"}
Oct 30 12:41:12.171311 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.171232    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-77dd99c685-ffnbm" event={"ID":"32d480e2-e304-4124-adf0-8cc0e31c9acf","Type":"ContainerDied","Data":"a79f1fb8104aede59320cdec65f51f51fa9d7804c57f9dc9784060c9c2a78610"}
Oct 30 12:41:12.171311 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.171266    4945 scope.go:122] "RemoveContainer" containerID="60268e39f39b8e02c74ae9839b7f367f15cc95db3dd119c221ec9bae64f1f81f"
Oct 30 12:41:12.172303 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.172280    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32d480e2-e304-4124-adf0-8cc0e31c9acf-kube-api-access-c9dsr" pod "32d480e2-e304-4124-adf0-8cc0e31c9acf" (UID: "32d480e2-e304-4124-adf0-8cc0e31c9acf"). InnerVolumeSpecName "kube-api-access-c9dsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:41:12.184175 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.184150    4945 scope.go:122] "RemoveContainer" containerID="60268e39f39b8e02c74ae9839b7f367f15cc95db3dd119c221ec9bae64f1f81f"
Oct 30 12:41:12.184452 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:41:12.184432    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60268e39f39b8e02c74ae9839b7f367f15cc95db3dd119c221ec9bae64f1f81f\": container with ID starting with 60268e39f39b8e02c74ae9839b7f367f15cc95db3dd119c221ec9bae64f1f81f not found: ID does not exist" containerID="60268e39f39b8e02c74ae9839b7f367f15cc95db3dd119c221ec9bae64f1f81f"
Oct 30 12:41:12.184506 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.184461    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60268e39f39b8e02c74ae9839b7f367f15cc95db3dd119c221ec9bae64f1f81f"} err="failed to get container status \"60268e39f39b8e02c74ae9839b7f367f15cc95db3dd119c221ec9bae64f1f81f\": rpc error: code = NotFound desc = could not find container \"60268e39f39b8e02c74ae9839b7f367f15cc95db3dd119c221ec9bae64f1f81f\": container with ID starting with 60268e39f39b8e02c74ae9839b7f367f15cc95db3dd119c221ec9bae64f1f81f not found: ID does not exist"
Oct 30 12:41:12.270958 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.270930    4945 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/32d480e2-e304-4124-adf0-8cc0e31c9acf-config-file\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:41:12.270958 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.270956    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9dsr\" (UniqueName: \"kubernetes.io/projected/32d480e2-e304-4124-adf0-8cc0e31c9acf-kube-api-access-c9dsr\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:41:12.513895 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.513861    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-77dd99c685-ffnbm"]
Oct 30 12:41:12.525337 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:12.525310    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-77dd99c685-ffnbm"]
Oct 30 12:41:13.547132 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:13.547102    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32d480e2-e304-4124-adf0-8cc0e31c9acf" path="/var/lib/kubelet/pods/32d480e2-e304-4124-adf0-8cc0e31c9acf/volumes"
Oct 30 12:41:16.966434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:16.966397    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz"]
Oct 30 12:41:16.966826 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:16.966803    4945 cpu_manager.go:414] "containerMap: removing container" podUID="32d480e2-e304-4124-adf0-8cc0e31c9acf" containerName="limitador"
Oct 30 12:41:16.966826 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:16.966816    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32d480e2-e304-4124-adf0-8cc0e31c9acf" containerName="limitador"
Oct 30 12:41:16.966901 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:16.966888    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="32d480e2-e304-4124-adf0-8cc0e31c9acf" containerName="limitador"
Oct 30 12:41:16.971204 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:16.971187    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz"
Oct 30 12:41:16.988250 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:16.988224    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz"]
Oct 30 12:41:16.997791 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:16.997762    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 12:41:16.997791 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:16.997776    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 12:41:17.031207 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:17.031177    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 12:41:17.031207 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:17.031188    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-nm9qh\""
Oct 30 12:41:17.110713 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:17.110680    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxndk\" (UniqueName: \"kubernetes.io/projected/8e1934f7-aafd-4e61-8f49-4a530f77403d-kube-api-access-sxndk\") pod \"postgres-7cdbf6dc44-cnwvz\" (UID: \"8e1934f7-aafd-4e61-8f49-4a530f77403d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz"
Oct 30 12:41:17.110913 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:17.110729    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8e1934f7-aafd-4e61-8f49-4a530f77403d-data\") pod \"postgres-7cdbf6dc44-cnwvz\" (UID: \"8e1934f7-aafd-4e61-8f49-4a530f77403d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz"
Oct 30 12:41:17.211973 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:17.211944    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxndk\" (UniqueName: \"kubernetes.io/projected/8e1934f7-aafd-4e61-8f49-4a530f77403d-kube-api-access-sxndk\") pod \"postgres-7cdbf6dc44-cnwvz\" (UID: \"8e1934f7-aafd-4e61-8f49-4a530f77403d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz"
Oct 30 12:41:17.212139 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:17.211985    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8e1934f7-aafd-4e61-8f49-4a530f77403d-data\") pod \"postgres-7cdbf6dc44-cnwvz\" (UID: \"8e1934f7-aafd-4e61-8f49-4a530f77403d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz"
Oct 30 12:41:17.212319 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:17.212302    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/8e1934f7-aafd-4e61-8f49-4a530f77403d-data\") pod \"postgres-7cdbf6dc44-cnwvz\" (UID: \"8e1934f7-aafd-4e61-8f49-4a530f77403d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz"
Oct 30 12:41:17.231255 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:17.231184    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxndk\" (UniqueName: \"kubernetes.io/projected/8e1934f7-aafd-4e61-8f49-4a530f77403d-kube-api-access-sxndk\") pod \"postgres-7cdbf6dc44-cnwvz\" (UID: \"8e1934f7-aafd-4e61-8f49-4a530f77403d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz"
Oct 30 12:41:17.283489 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:17.283460    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz"
Oct 30 12:41:17.575510 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:17.575328    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz"]
Oct 30 12:41:17.580075 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:41:17.580040    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e1934f7_aafd_4e61_8f49_4a530f77403d.slice/crio-7920464b66928c9e9ad31c20152291df1709aaafa37ba0305e6904211bc7b0f8 WatchSource:0}: Error finding container 7920464b66928c9e9ad31c20152291df1709aaafa37ba0305e6904211bc7b0f8: Status 404 returned error can't find the container with id 7920464b66928c9e9ad31c20152291df1709aaafa37ba0305e6904211bc7b0f8
Oct 30 12:41:17.581462 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:17.581439    4945 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 12:41:18.209435 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:18.209391    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz" event={"ID":"8e1934f7-aafd-4e61-8f49-4a530f77403d","Type":"ContainerStarted","Data":"7920464b66928c9e9ad31c20152291df1709aaafa37ba0305e6904211bc7b0f8"}
Oct 30 12:41:23.124345 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:23.124312    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-s9qks"
Oct 30 12:41:23.245693 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:23.245642    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz" event={"ID":"8e1934f7-aafd-4e61-8f49-4a530f77403d","Type":"ContainerStarted","Data":"4f927ac19cd85ee03489f90c939b8cf1d74306c55ebaa8a079e6bcb4a39bcde7"}
Oct 30 12:41:23.245874 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:23.245717    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz"
Oct 30 12:41:23.285852 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:23.285801    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz" podStartSLOduration=2.601641698 podStartE2EDuration="7.285786753s" totalImagesPullingTime="4.684145055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:41:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:41:23.284328364 +0000 UTC m=+327.862277453" watchObservedRunningTime="2026-10-30 12:41:23.285786753 +0000 UTC m=+327.863735841"
Oct 30 12:41:26.189327 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:41:26.189286    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:41:29.280667 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:29.280633    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cnwvz"
Oct 30 12:41:36.252869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.252828    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c4696569f-ffktb"]
Oct 30 12:41:36.281909 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.281877    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c4696569f-ffktb"]
Oct 30 12:41:36.282077 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.281977    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c4696569f-ffktb"
Oct 30 12:41:36.298814 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.298787    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 12:41:36.309964 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.309923    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 12:41:36.310107 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.309968    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jdbh2\""
Oct 30 12:41:36.310107 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.310008    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 12:41:36.384335 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.384297    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ea767e4-ae53-4d16-8414-52517c903fd0-metrics-certs\") pod \"maas-controller-c4696569f-ffktb\" (UID: \"9ea767e4-ae53-4d16-8414-52517c903fd0\") " pod="opendatahub/maas-controller-c4696569f-ffktb"
Oct 30 12:41:36.384501 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.384351    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ea767e4-ae53-4d16-8414-52517c903fd0-webhook-cert\") pod \"maas-controller-c4696569f-ffktb\" (UID: \"9ea767e4-ae53-4d16-8414-52517c903fd0\") " pod="opendatahub/maas-controller-c4696569f-ffktb"
Oct 30 12:41:36.384501 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.384374    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76f9j\" (UniqueName: \"kubernetes.io/projected/9ea767e4-ae53-4d16-8414-52517c903fd0-kube-api-access-76f9j\") pod \"maas-controller-c4696569f-ffktb\" (UID: \"9ea767e4-ae53-4d16-8414-52517c903fd0\") " pod="opendatahub/maas-controller-c4696569f-ffktb"
Oct 30 12:41:36.485612 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.485572    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ea767e4-ae53-4d16-8414-52517c903fd0-metrics-certs\") pod \"maas-controller-c4696569f-ffktb\" (UID: \"9ea767e4-ae53-4d16-8414-52517c903fd0\") " pod="opendatahub/maas-controller-c4696569f-ffktb"
Oct 30 12:41:36.485810 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.485670    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ea767e4-ae53-4d16-8414-52517c903fd0-webhook-cert\") pod \"maas-controller-c4696569f-ffktb\" (UID: \"9ea767e4-ae53-4d16-8414-52517c903fd0\") " pod="opendatahub/maas-controller-c4696569f-ffktb"
Oct 30 12:41:36.485810 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.485700    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76f9j\" (UniqueName: \"kubernetes.io/projected/9ea767e4-ae53-4d16-8414-52517c903fd0-kube-api-access-76f9j\") pod \"maas-controller-c4696569f-ffktb\" (UID: \"9ea767e4-ae53-4d16-8414-52517c903fd0\") " pod="opendatahub/maas-controller-c4696569f-ffktb"
Oct 30 12:41:36.488109 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.488078    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ea767e4-ae53-4d16-8414-52517c903fd0-webhook-cert\") pod \"maas-controller-c4696569f-ffktb\" (UID: \"9ea767e4-ae53-4d16-8414-52517c903fd0\") " pod="opendatahub/maas-controller-c4696569f-ffktb"
Oct 30 12:41:36.488109 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.488106    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ea767e4-ae53-4d16-8414-52517c903fd0-metrics-certs\") pod \"maas-controller-c4696569f-ffktb\" (UID: \"9ea767e4-ae53-4d16-8414-52517c903fd0\") " pod="opendatahub/maas-controller-c4696569f-ffktb"
Oct 30 12:41:36.504323 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.504242    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76f9j\" (UniqueName: \"kubernetes.io/projected/9ea767e4-ae53-4d16-8414-52517c903fd0-kube-api-access-76f9j\") pod \"maas-controller-c4696569f-ffktb\" (UID: \"9ea767e4-ae53-4d16-8414-52517c903fd0\") " pod="opendatahub/maas-controller-c4696569f-ffktb"
Oct 30 12:41:36.597145 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.597109    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c4696569f-ffktb"
Oct 30 12:41:36.757222 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.757147    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5d7d58c6ff-pncqs"]
Oct 30 12:41:36.762156 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.762134    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d7d58c6ff-pncqs"
Oct 30 12:41:36.782700 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.782669    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d7d58c6ff-pncqs"]
Oct 30 12:41:36.889034 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.888995    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tt2b\" (UniqueName: \"kubernetes.io/projected/180c4327-13ad-48b0-b5f1-c193ab9b0c9d-kube-api-access-7tt2b\") pod \"maas-controller-5d7d58c6ff-pncqs\" (UID: \"180c4327-13ad-48b0-b5f1-c193ab9b0c9d\") " pod="opendatahub/maas-controller-5d7d58c6ff-pncqs"
Oct 30 12:41:36.889364 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.889338    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/180c4327-13ad-48b0-b5f1-c193ab9b0c9d-metrics-certs\") pod \"maas-controller-5d7d58c6ff-pncqs\" (UID: \"180c4327-13ad-48b0-b5f1-c193ab9b0c9d\") " pod="opendatahub/maas-controller-5d7d58c6ff-pncqs"
Oct 30 12:41:36.889510 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.889389    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/180c4327-13ad-48b0-b5f1-c193ab9b0c9d-webhook-cert\") pod \"maas-controller-5d7d58c6ff-pncqs\" (UID: \"180c4327-13ad-48b0-b5f1-c193ab9b0c9d\") " pod="opendatahub/maas-controller-5d7d58c6ff-pncqs"
Oct 30 12:41:36.890547 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.890513    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c4696569f-ffktb"]
Oct 30 12:41:36.894343 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:41:36.894311    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ea767e4_ae53_4d16_8414_52517c903fd0.slice/crio-3bffb9cebaa38ea37b1aba7b78de8fbcaf7ca56bb0f6410f055b005949486a5c WatchSource:0}: Error finding container 3bffb9cebaa38ea37b1aba7b78de8fbcaf7ca56bb0f6410f055b005949486a5c: Status 404 returned error can't find the container with id 3bffb9cebaa38ea37b1aba7b78de8fbcaf7ca56bb0f6410f055b005949486a5c
Oct 30 12:41:36.990248 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.990209    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tt2b\" (UniqueName: \"kubernetes.io/projected/180c4327-13ad-48b0-b5f1-c193ab9b0c9d-kube-api-access-7tt2b\") pod \"maas-controller-5d7d58c6ff-pncqs\" (UID: \"180c4327-13ad-48b0-b5f1-c193ab9b0c9d\") " pod="opendatahub/maas-controller-5d7d58c6ff-pncqs"
Oct 30 12:41:36.990449 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.990288    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/180c4327-13ad-48b0-b5f1-c193ab9b0c9d-metrics-certs\") pod \"maas-controller-5d7d58c6ff-pncqs\" (UID: \"180c4327-13ad-48b0-b5f1-c193ab9b0c9d\") " pod="opendatahub/maas-controller-5d7d58c6ff-pncqs"
Oct 30 12:41:36.990449 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.990335    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/180c4327-13ad-48b0-b5f1-c193ab9b0c9d-webhook-cert\") pod \"maas-controller-5d7d58c6ff-pncqs\" (UID: \"180c4327-13ad-48b0-b5f1-c193ab9b0c9d\") " pod="opendatahub/maas-controller-5d7d58c6ff-pncqs"
Oct 30 12:41:36.992941 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.992915    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/180c4327-13ad-48b0-b5f1-c193ab9b0c9d-webhook-cert\") pod \"maas-controller-5d7d58c6ff-pncqs\" (UID: \"180c4327-13ad-48b0-b5f1-c193ab9b0c9d\") " pod="opendatahub/maas-controller-5d7d58c6ff-pncqs"
Oct 30 12:41:36.993103 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:36.993077    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/180c4327-13ad-48b0-b5f1-c193ab9b0c9d-metrics-certs\") pod \"maas-controller-5d7d58c6ff-pncqs\" (UID: \"180c4327-13ad-48b0-b5f1-c193ab9b0c9d\") " pod="opendatahub/maas-controller-5d7d58c6ff-pncqs"
Oct 30 12:41:37.008923 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:37.008853    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tt2b\" (UniqueName: \"kubernetes.io/projected/180c4327-13ad-48b0-b5f1-c193ab9b0c9d-kube-api-access-7tt2b\") pod \"maas-controller-5d7d58c6ff-pncqs\" (UID: \"180c4327-13ad-48b0-b5f1-c193ab9b0c9d\") " pod="opendatahub/maas-controller-5d7d58c6ff-pncqs"
Oct 30 12:41:37.076445 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:37.076409    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d7d58c6ff-pncqs"
Oct 30 12:41:37.318668 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:37.318573    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c4696569f-ffktb" event={"ID":"9ea767e4-ae53-4d16-8414-52517c903fd0","Type":"ContainerStarted","Data":"3bffb9cebaa38ea37b1aba7b78de8fbcaf7ca56bb0f6410f055b005949486a5c"}
Oct 30 12:41:37.368893 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:37.368862    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d7d58c6ff-pncqs"]
Oct 30 12:41:37.371906 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:41:37.371880    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod180c4327_13ad_48b0_b5f1_c193ab9b0c9d.slice/crio-189b4ed48f80346cdb64580fd647473958684fdc158137295f70166a27aa06f7 WatchSource:0}: Error finding container 189b4ed48f80346cdb64580fd647473958684fdc158137295f70166a27aa06f7: Status 404 returned error can't find the container with id 189b4ed48f80346cdb64580fd647473958684fdc158137295f70166a27aa06f7
Oct 30 12:41:38.334083 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:38.334040    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d7d58c6ff-pncqs" event={"ID":"180c4327-13ad-48b0-b5f1-c193ab9b0c9d","Type":"ContainerStarted","Data":"189b4ed48f80346cdb64580fd647473958684fdc158137295f70166a27aa06f7"}
Oct 30 12:41:40.347552 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:40.347468    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d7d58c6ff-pncqs" event={"ID":"180c4327-13ad-48b0-b5f1-c193ab9b0c9d","Type":"ContainerStarted","Data":"eabc9f9ffee979a4a8ac0527a248557cbd7884ec9c0638379ba5e4794d3c50fa"}
Oct 30 12:41:40.348017 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:40.347587    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5d7d58c6ff-pncqs"
Oct 30 12:41:40.349101 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:40.349077    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c4696569f-ffktb" event={"ID":"9ea767e4-ae53-4d16-8414-52517c903fd0","Type":"ContainerStarted","Data":"d5cbde7bc3e607894de1a20ac7fca3ef917404a28b7bc2dc767d9a5d0790a5e8"}
Oct 30 12:41:40.349237 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:40.349223    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c4696569f-ffktb"
Oct 30 12:41:40.390951 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:40.390900    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-5d7d58c6ff-pncqs" podStartSLOduration=1.717472798 podStartE2EDuration="4.390881935s" totalImagesPullingTime="2.673409137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:41:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:41:40.388130253 +0000 UTC m=+344.966079342" watchObservedRunningTime="2026-10-30 12:41:40.390881935 +0000 UTC m=+344.968831051"
Oct 30 12:41:40.430234 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:40.430182    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-c4696569f-ffktb" podStartSLOduration=1.545284694 podStartE2EDuration="4.43016829s" totalImagesPullingTime="2.884883596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:41:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:41:40.428214792 +0000 UTC m=+345.006163881" watchObservedRunningTime="2026-10-30 12:41:40.43016829 +0000 UTC m=+345.008117378"
Oct 30 12:41:41.452208 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.452171    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"]
Oct 30 12:41:41.455794 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.455774    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:41:41.475909 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.475884    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 12:41:41.483016 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.482964    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 12:41:41.483016 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.482969    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 12:41:41.483206 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.483074    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-9nnxp\""
Oct 30 12:41:41.486887 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.486865    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"]
Oct 30 12:41:41.530259 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.530225    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg4zq\" (UniqueName: \"kubernetes.io/projected/5745d316-11ab-412b-8e86-e3068d830cb7-kube-api-access-xg4zq\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-ktflf\" (UID: \"5745d316-11ab-412b-8e86-e3068d830cb7\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:41:41.530409 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.530283    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5745d316-11ab-412b-8e86-e3068d830cb7-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-ktflf\" (UID: \"5745d316-11ab-412b-8e86-e3068d830cb7\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:41:41.530409 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.530377    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5745d316-11ab-412b-8e86-e3068d830cb7-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-ktflf\" (UID: \"5745d316-11ab-412b-8e86-e3068d830cb7\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:41:41.530409 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.530401    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/5745d316-11ab-412b-8e86-e3068d830cb7-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-ktflf\" (UID: \"5745d316-11ab-412b-8e86-e3068d830cb7\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:41:41.631253 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.631223    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5745d316-11ab-412b-8e86-e3068d830cb7-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-ktflf\" (UID: \"5745d316-11ab-412b-8e86-e3068d830cb7\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:41:41.631253 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.631258    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5745d316-11ab-412b-8e86-e3068d830cb7-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-ktflf\" (UID: \"5745d316-11ab-412b-8e86-e3068d830cb7\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:41:41.631505 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.631280    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/5745d316-11ab-412b-8e86-e3068d830cb7-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-ktflf\" (UID: \"5745d316-11ab-412b-8e86-e3068d830cb7\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:41:41.631505 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.631338    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg4zq\" (UniqueName: \"kubernetes.io/projected/5745d316-11ab-412b-8e86-e3068d830cb7-kube-api-access-xg4zq\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-ktflf\" (UID: \"5745d316-11ab-412b-8e86-e3068d830cb7\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:41:41.631922 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.631896    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5745d316-11ab-412b-8e86-e3068d830cb7-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-ktflf\" (UID: \"5745d316-11ab-412b-8e86-e3068d830cb7\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:41:41.632218 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.632196    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/5745d316-11ab-412b-8e86-e3068d830cb7-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-ktflf\" (UID: \"5745d316-11ab-412b-8e86-e3068d830cb7\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:41:41.633801 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.633780    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5745d316-11ab-412b-8e86-e3068d830cb7-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-ktflf\" (UID: \"5745d316-11ab-412b-8e86-e3068d830cb7\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:41:41.649775 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.649745    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg4zq\" (UniqueName: \"kubernetes.io/projected/5745d316-11ab-412b-8e86-e3068d830cb7-kube-api-access-xg4zq\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-ktflf\" (UID: \"5745d316-11ab-412b-8e86-e3068d830cb7\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:41:41.768500 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:41.768406    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:41:42.066379 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:42.066294    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"]
Oct 30 12:41:42.069489 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:41:42.069460    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5745d316_11ab_412b_8e86_e3068d830cb7.slice/crio-aa2b3b132282da451393fea9d5e7a440e4e755893498006c64a57247a85019b2 WatchSource:0}: Error finding container aa2b3b132282da451393fea9d5e7a440e4e755893498006c64a57247a85019b2: Status 404 returned error can't find the container with id aa2b3b132282da451393fea9d5e7a440e4e755893498006c64a57247a85019b2
Oct 30 12:41:42.361690 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:42.361574    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf" event={"ID":"5745d316-11ab-412b-8e86-e3068d830cb7","Type":"ContainerStarted","Data":"aa2b3b132282da451393fea9d5e7a440e4e755893498006c64a57247a85019b2"}
Oct 30 12:41:51.358226 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:51.358192    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5d7d58c6ff-pncqs"
Oct 30 12:41:51.358732 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:51.358637    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c4696569f-ffktb"
Oct 30 12:41:51.443831 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:51.443712    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c4696569f-ffktb"]
Oct 30 12:41:51.448117 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:51.448082    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-c4696569f-ffktb" podUID="9ea767e4-ae53-4d16-8414-52517c903fd0" containerName="manager" containerID="cri-o://d5cbde7bc3e607894de1a20ac7fca3ef917404a28b7bc2dc767d9a5d0790a5e8" gracePeriod=30
Oct 30 12:41:51.721273 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:51.720999    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c4696569f-ffktb"
Oct 30 12:41:51.846476 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:51.846438    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9ea767e4-ae53-4d16-8414-52517c903fd0-webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ea767e4-ae53-4d16-8414-52517c903fd0-webhook-cert\") pod \"9ea767e4-ae53-4d16-8414-52517c903fd0\" (UID: \"9ea767e4-ae53-4d16-8414-52517c903fd0\") "
Oct 30 12:41:51.846682 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:51.846536    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ea767e4-ae53-4d16-8414-52517c903fd0-kube-api-access-76f9j\" (UniqueName: \"kubernetes.io/projected/9ea767e4-ae53-4d16-8414-52517c903fd0-kube-api-access-76f9j\") pod \"9ea767e4-ae53-4d16-8414-52517c903fd0\" (UID: \"9ea767e4-ae53-4d16-8414-52517c903fd0\") "
Oct 30 12:41:51.846682 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:51.846667    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9ea767e4-ae53-4d16-8414-52517c903fd0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ea767e4-ae53-4d16-8414-52517c903fd0-metrics-certs\") pod \"9ea767e4-ae53-4d16-8414-52517c903fd0\" (UID: \"9ea767e4-ae53-4d16-8414-52517c903fd0\") "
Oct 30 12:41:51.849298 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:51.849260    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ea767e4-ae53-4d16-8414-52517c903fd0-webhook-cert" pod "9ea767e4-ae53-4d16-8414-52517c903fd0" (UID: "9ea767e4-ae53-4d16-8414-52517c903fd0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:41:51.866981 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:51.866887    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ea767e4-ae53-4d16-8414-52517c903fd0-kube-api-access-76f9j" pod "9ea767e4-ae53-4d16-8414-52517c903fd0" (UID: "9ea767e4-ae53-4d16-8414-52517c903fd0"). InnerVolumeSpecName "kube-api-access-76f9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:41:51.867109 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:51.866987    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ea767e4-ae53-4d16-8414-52517c903fd0-metrics-certs" pod "9ea767e4-ae53-4d16-8414-52517c903fd0" (UID: "9ea767e4-ae53-4d16-8414-52517c903fd0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:41:51.947805 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:51.947774    4945 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9ea767e4-ae53-4d16-8414-52517c903fd0-webhook-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:41:51.947805 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:51.947807    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76f9j\" (UniqueName: \"kubernetes.io/projected/9ea767e4-ae53-4d16-8414-52517c903fd0-kube-api-access-76f9j\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:41:51.948012 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:51.947821    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ea767e4-ae53-4d16-8414-52517c903fd0-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:41:52.454201 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:52.454168    4945 generic.go:356] "Generic (PLEG): container finished" podID="9ea767e4-ae53-4d16-8414-52517c903fd0" containerID="d5cbde7bc3e607894de1a20ac7fca3ef917404a28b7bc2dc767d9a5d0790a5e8" exitCode=0
Oct 30 12:41:52.454644 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:52.454216    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c4696569f-ffktb"
Oct 30 12:41:52.454644 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:52.454248    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c4696569f-ffktb" event={"ID":"9ea767e4-ae53-4d16-8414-52517c903fd0","Type":"ContainerDied","Data":"d5cbde7bc3e607894de1a20ac7fca3ef917404a28b7bc2dc767d9a5d0790a5e8"}
Oct 30 12:41:52.454644 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:52.454311    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c4696569f-ffktb" event={"ID":"9ea767e4-ae53-4d16-8414-52517c903fd0","Type":"ContainerDied","Data":"3bffb9cebaa38ea37b1aba7b78de8fbcaf7ca56bb0f6410f055b005949486a5c"}
Oct 30 12:41:52.454644 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:52.454336    4945 scope.go:122] "RemoveContainer" containerID="d5cbde7bc3e607894de1a20ac7fca3ef917404a28b7bc2dc767d9a5d0790a5e8"
Oct 30 12:41:52.463869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:52.463852    4945 scope.go:122] "RemoveContainer" containerID="d5cbde7bc3e607894de1a20ac7fca3ef917404a28b7bc2dc767d9a5d0790a5e8"
Oct 30 12:41:52.464144 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:41:52.464125    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5cbde7bc3e607894de1a20ac7fca3ef917404a28b7bc2dc767d9a5d0790a5e8\": container with ID starting with d5cbde7bc3e607894de1a20ac7fca3ef917404a28b7bc2dc767d9a5d0790a5e8 not found: ID does not exist" containerID="d5cbde7bc3e607894de1a20ac7fca3ef917404a28b7bc2dc767d9a5d0790a5e8"
Oct 30 12:41:52.464207 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:52.464152    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5cbde7bc3e607894de1a20ac7fca3ef917404a28b7bc2dc767d9a5d0790a5e8"} err="failed to get container status \"d5cbde7bc3e607894de1a20ac7fca3ef917404a28b7bc2dc767d9a5d0790a5e8\": rpc error: code = NotFound desc = could not find container \"d5cbde7bc3e607894de1a20ac7fca3ef917404a28b7bc2dc767d9a5d0790a5e8\": container with ID starting with d5cbde7bc3e607894de1a20ac7fca3ef917404a28b7bc2dc767d9a5d0790a5e8 not found: ID does not exist"
Oct 30 12:41:52.518965 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:52.518933    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-c4696569f-ffktb"]
Oct 30 12:41:52.530123 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:52.530096    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-c4696569f-ffktb"]
Oct 30 12:41:53.460793 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:53.460751    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf" event={"ID":"5745d316-11ab-412b-8e86-e3068d830cb7","Type":"ContainerStarted","Data":"e93fb8a49a9ec145f76227a932d82f2dcb3f985e3000e6d5bb9108b824d398bb"}
Oct 30 12:41:53.461240 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:53.460898    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:41:53.507860 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:53.507810    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf" podStartSLOduration=1.9738235720000001 podStartE2EDuration="12.507797455s" totalImagesPullingTime="10.533973883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:41:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:41:53.504553822 +0000 UTC m=+358.082502910" watchObservedRunningTime="2026-10-30 12:41:53.507797455 +0000 UTC m=+358.085746544"
Oct 30 12:41:53.547780 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:53.547750    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ea767e4-ae53-4d16-8414-52517c903fd0" path="/var/lib/kubelet/pods/9ea767e4-ae53-4d16-8414-52517c903fd0/volumes"
Oct 30 12:41:56.191053 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:41:56.191008    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:41:57.031138 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.031104    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-5k947"]
Oct 30 12:41:57.031518 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.031505    4945 cpu_manager.go:414] "containerMap: removing container" podUID="9ea767e4-ae53-4d16-8414-52517c903fd0" containerName="manager"
Oct 30 12:41:57.031563 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.031519    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9ea767e4-ae53-4d16-8414-52517c903fd0" containerName="manager"
Oct 30 12:41:57.031621 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.031590    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="9ea767e4-ae53-4d16-8414-52517c903fd0" containerName="manager"
Oct 30 12:41:57.037960 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.037937    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-5k947"
Oct 30 12:41:57.053825 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.053800    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-5k947"]
Oct 30 12:41:57.063284 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.063265    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Oct 30 12:41:57.063370 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.063342    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-r8zgt\""
Oct 30 12:41:57.077045 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.077012    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"]
Oct 30 12:41:57.081848 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.081824    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"
Oct 30 12:41:57.092414 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.092388    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55710abf-0a0a-4830-a4dd-54083ab2ac04-cert\") pod \"kserve-controller-manager-776cc66d47-5k947\" (UID: \"55710abf-0a0a-4830-a4dd-54083ab2ac04\") " pod="opendatahub/kserve-controller-manager-776cc66d47-5k947"
Oct 30 12:41:57.092502 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.092458    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfbv6\" (UniqueName: \"kubernetes.io/projected/55710abf-0a0a-4830-a4dd-54083ab2ac04-kube-api-access-jfbv6\") pod \"kserve-controller-manager-776cc66d47-5k947\" (UID: \"55710abf-0a0a-4830-a4dd-54083ab2ac04\") " pod="opendatahub/kserve-controller-manager-776cc66d47-5k947"
Oct 30 12:41:57.102210 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.102188    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"]
Oct 30 12:41:57.120234 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.120207    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-gzpff"]
Oct 30 12:41:57.130732 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.130702    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-gzpff"
Oct 30 12:41:57.147901 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.147876    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-gzpff"]
Oct 30 12:41:57.153719 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.153700    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Oct 30 12:41:57.153992 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.153975    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-vsjb8\""
Oct 30 12:41:57.154077 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.154064    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Oct 30 12:41:57.154135 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.154066    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-t5hf9\""
Oct 30 12:41:57.169028 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.169001    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-n6rkn"]
Oct 30 12:41:57.173609 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.173556    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-n6rkn"
Oct 30 12:41:57.190749 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.190721    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-n6rkn"]
Oct 30 12:41:57.193473 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.193448    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b818f62c-1645-4782-93a6-da2341cdcce7-tls-certs\") pod \"model-serving-api-75bd89fdfc-gzpff\" (UID: \"b818f62c-1645-4782-93a6-da2341cdcce7\") " pod="opendatahub/model-serving-api-75bd89fdfc-gzpff"
Oct 30 12:41:57.193883 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.193485    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pdj4\" (UniqueName: \"kubernetes.io/projected/b818f62c-1645-4782-93a6-da2341cdcce7-kube-api-access-2pdj4\") pod \"model-serving-api-75bd89fdfc-gzpff\" (UID: \"b818f62c-1645-4782-93a6-da2341cdcce7\") " pod="opendatahub/model-serving-api-75bd89fdfc-gzpff"
Oct 30 12:41:57.193883 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.193542    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55710abf-0a0a-4830-a4dd-54083ab2ac04-cert\") pod \"kserve-controller-manager-776cc66d47-5k947\" (UID: \"55710abf-0a0a-4830-a4dd-54083ab2ac04\") " pod="opendatahub/kserve-controller-manager-776cc66d47-5k947"
Oct 30 12:41:57.193883 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.193679    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfbv6\" (UniqueName: \"kubernetes.io/projected/55710abf-0a0a-4830-a4dd-54083ab2ac04-kube-api-access-jfbv6\") pod \"kserve-controller-manager-776cc66d47-5k947\" (UID: \"55710abf-0a0a-4830-a4dd-54083ab2ac04\") " pod="opendatahub/kserve-controller-manager-776cc66d47-5k947"
Oct 30 12:41:57.193883 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.193762    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zbwc\" (UniqueName: \"kubernetes.io/projected/fed6ba8b-c3eb-444f-8def-0be29c763e81-kube-api-access-6zbwc\") pod \"llmisvc-controller-manager-869d5dcc5d-xvtl9\" (UID: \"fed6ba8b-c3eb-444f-8def-0be29c763e81\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"
Oct 30 12:41:57.193883 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.193797    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fed6ba8b-c3eb-444f-8def-0be29c763e81-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-xvtl9\" (UID: \"fed6ba8b-c3eb-444f-8def-0be29c763e81\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"
Oct 30 12:41:57.195988 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.195964    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/55710abf-0a0a-4830-a4dd-54083ab2ac04-cert\") pod \"kserve-controller-manager-776cc66d47-5k947\" (UID: \"55710abf-0a0a-4830-a4dd-54083ab2ac04\") " pod="opendatahub/kserve-controller-manager-776cc66d47-5k947"
Oct 30 12:41:57.215221 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.215199    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfbv6\" (UniqueName: \"kubernetes.io/projected/55710abf-0a0a-4830-a4dd-54083ab2ac04-kube-api-access-jfbv6\") pod \"kserve-controller-manager-776cc66d47-5k947\" (UID: \"55710abf-0a0a-4830-a4dd-54083ab2ac04\") " pod="opendatahub/kserve-controller-manager-776cc66d47-5k947"
Oct 30 12:41:57.254037 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.254010    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-zvccx\""
Oct 30 12:41:57.254037 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.254022    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Oct 30 12:41:57.254178 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.254054    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Oct 30 12:41:57.294624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.294556    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/703d6675-c844-48eb-a019-2a09a2123307-cert\") pod \"odh-model-controller-769875db56-n6rkn\" (UID: \"703d6675-c844-48eb-a019-2a09a2123307\") " pod="opendatahub/odh-model-controller-769875db56-n6rkn"
Oct 30 12:41:57.294624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.294591    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rb77\" (UniqueName: \"kubernetes.io/projected/703d6675-c844-48eb-a019-2a09a2123307-kube-api-access-2rb77\") pod \"odh-model-controller-769875db56-n6rkn\" (UID: \"703d6675-c844-48eb-a019-2a09a2123307\") " pod="opendatahub/odh-model-controller-769875db56-n6rkn"
Oct 30 12:41:57.294788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.294767    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zbwc\" (UniqueName: \"kubernetes.io/projected/fed6ba8b-c3eb-444f-8def-0be29c763e81-kube-api-access-6zbwc\") pod \"llmisvc-controller-manager-869d5dcc5d-xvtl9\" (UID: \"fed6ba8b-c3eb-444f-8def-0be29c763e81\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"
Oct 30 12:41:57.294826 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.294801    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fed6ba8b-c3eb-444f-8def-0be29c763e81-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-xvtl9\" (UID: \"fed6ba8b-c3eb-444f-8def-0be29c763e81\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"
Oct 30 12:41:57.294859 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.294827    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/703d6675-c844-48eb-a019-2a09a2123307-metrics-certs\") pod \"odh-model-controller-769875db56-n6rkn\" (UID: \"703d6675-c844-48eb-a019-2a09a2123307\") " pod="opendatahub/odh-model-controller-769875db56-n6rkn"
Oct 30 12:41:57.294902 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.294856    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b818f62c-1645-4782-93a6-da2341cdcce7-tls-certs\") pod \"model-serving-api-75bd89fdfc-gzpff\" (UID: \"b818f62c-1645-4782-93a6-da2341cdcce7\") " pod="opendatahub/model-serving-api-75bd89fdfc-gzpff"
Oct 30 12:41:57.294902 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.294877    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pdj4\" (UniqueName: \"kubernetes.io/projected/b818f62c-1645-4782-93a6-da2341cdcce7-kube-api-access-2pdj4\") pod \"model-serving-api-75bd89fdfc-gzpff\" (UID: \"b818f62c-1645-4782-93a6-da2341cdcce7\") " pod="opendatahub/model-serving-api-75bd89fdfc-gzpff"
Oct 30 12:41:57.297071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.297049    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fed6ba8b-c3eb-444f-8def-0be29c763e81-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-xvtl9\" (UID: \"fed6ba8b-c3eb-444f-8def-0be29c763e81\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"
Oct 30 12:41:57.297213 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.297193    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b818f62c-1645-4782-93a6-da2341cdcce7-tls-certs\") pod \"model-serving-api-75bd89fdfc-gzpff\" (UID: \"b818f62c-1645-4782-93a6-da2341cdcce7\") " pod="opendatahub/model-serving-api-75bd89fdfc-gzpff"
Oct 30 12:41:57.314009 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.313985    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pdj4\" (UniqueName: \"kubernetes.io/projected/b818f62c-1645-4782-93a6-da2341cdcce7-kube-api-access-2pdj4\") pod \"model-serving-api-75bd89fdfc-gzpff\" (UID: \"b818f62c-1645-4782-93a6-da2341cdcce7\") " pod="opendatahub/model-serving-api-75bd89fdfc-gzpff"
Oct 30 12:41:57.314102 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.314087    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zbwc\" (UniqueName: \"kubernetes.io/projected/fed6ba8b-c3eb-444f-8def-0be29c763e81-kube-api-access-6zbwc\") pod \"llmisvc-controller-manager-869d5dcc5d-xvtl9\" (UID: \"fed6ba8b-c3eb-444f-8def-0be29c763e81\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"
Oct 30 12:41:57.351503 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.351472    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-5k947"
Oct 30 12:41:57.394631 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.394586    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"
Oct 30 12:41:57.395881 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.395856    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/703d6675-c844-48eb-a019-2a09a2123307-cert\") pod \"odh-model-controller-769875db56-n6rkn\" (UID: \"703d6675-c844-48eb-a019-2a09a2123307\") " pod="opendatahub/odh-model-controller-769875db56-n6rkn"
Oct 30 12:41:57.395989 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.395903    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rb77\" (UniqueName: \"kubernetes.io/projected/703d6675-c844-48eb-a019-2a09a2123307-kube-api-access-2rb77\") pod \"odh-model-controller-769875db56-n6rkn\" (UID: \"703d6675-c844-48eb-a019-2a09a2123307\") " pod="opendatahub/odh-model-controller-769875db56-n6rkn"
Oct 30 12:41:57.396086 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.396070    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/703d6675-c844-48eb-a019-2a09a2123307-metrics-certs\") pod \"odh-model-controller-769875db56-n6rkn\" (UID: \"703d6675-c844-48eb-a019-2a09a2123307\") " pod="opendatahub/odh-model-controller-769875db56-n6rkn"
Oct 30 12:41:57.398385 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.398362    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/703d6675-c844-48eb-a019-2a09a2123307-metrics-certs\") pod \"odh-model-controller-769875db56-n6rkn\" (UID: \"703d6675-c844-48eb-a019-2a09a2123307\") " pod="opendatahub/odh-model-controller-769875db56-n6rkn"
Oct 30 12:41:57.398465 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.398409    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/703d6675-c844-48eb-a019-2a09a2123307-cert\") pod \"odh-model-controller-769875db56-n6rkn\" (UID: \"703d6675-c844-48eb-a019-2a09a2123307\") " pod="opendatahub/odh-model-controller-769875db56-n6rkn"
Oct 30 12:41:57.436400 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.436360    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rb77\" (UniqueName: \"kubernetes.io/projected/703d6675-c844-48eb-a019-2a09a2123307-kube-api-access-2rb77\") pod \"odh-model-controller-769875db56-n6rkn\" (UID: \"703d6675-c844-48eb-a019-2a09a2123307\") " pod="opendatahub/odh-model-controller-769875db56-n6rkn"
Oct 30 12:41:57.444297 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.444103    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-gzpff"
Oct 30 12:41:57.491282 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.490987    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-n6rkn"
Oct 30 12:41:57.649354 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.649332    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-5k947"]
Oct 30 12:41:57.652198 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:41:57.652160    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55710abf_0a0a_4830_a4dd_54083ab2ac04.slice/crio-1adb82688036cc1e51a3b02c0baf70844b545d800e020b02db037f2d29c21ae7 WatchSource:0}: Error finding container 1adb82688036cc1e51a3b02c0baf70844b545d800e020b02db037f2d29c21ae7: Status 404 returned error can't find the container with id 1adb82688036cc1e51a3b02c0baf70844b545d800e020b02db037f2d29c21ae7
Oct 30 12:41:57.725249 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.725214    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"]
Oct 30 12:41:57.728314 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:41:57.728290    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfed6ba8b_c3eb_444f_8def_0be29c763e81.slice/crio-6805d1ae96f9b8830012b109e007afd2c11d1277640332cc84814ee5e411e2fb WatchSource:0}: Error finding container 6805d1ae96f9b8830012b109e007afd2c11d1277640332cc84814ee5e411e2fb: Status 404 returned error can't find the container with id 6805d1ae96f9b8830012b109e007afd2c11d1277640332cc84814ee5e411e2fb
Oct 30 12:41:57.749021 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.748951    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-gzpff"]
Oct 30 12:41:57.751373 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:41:57.751348    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb818f62c_1645_4782_93a6_da2341cdcce7.slice/crio-7a574de5ed4e2428a947caba8b3a4e90d6232998b675c5962acfb635665652fe WatchSource:0}: Error finding container 7a574de5ed4e2428a947caba8b3a4e90d6232998b675c5962acfb635665652fe: Status 404 returned error can't find the container with id 7a574de5ed4e2428a947caba8b3a4e90d6232998b675c5962acfb635665652fe
Oct 30 12:41:57.800978 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:57.800950    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-n6rkn"]
Oct 30 12:41:57.803930 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:41:57.803903    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod703d6675_c844_48eb_a019_2a09a2123307.slice/crio-268ee96708a1bdb4dff05e02776e5c2315bffa635f661021fa8599b6e8ebe8fa WatchSource:0}: Error finding container 268ee96708a1bdb4dff05e02776e5c2315bffa635f661021fa8599b6e8ebe8fa: Status 404 returned error can't find the container with id 268ee96708a1bdb4dff05e02776e5c2315bffa635f661021fa8599b6e8ebe8fa
Oct 30 12:41:58.502494 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:58.502451    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-gzpff" event={"ID":"b818f62c-1645-4782-93a6-da2341cdcce7","Type":"ContainerStarted","Data":"7a574de5ed4e2428a947caba8b3a4e90d6232998b675c5962acfb635665652fe"}
Oct 30 12:41:58.504273 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:58.504218    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-n6rkn" event={"ID":"703d6675-c844-48eb-a019-2a09a2123307","Type":"ContainerStarted","Data":"268ee96708a1bdb4dff05e02776e5c2315bffa635f661021fa8599b6e8ebe8fa"}
Oct 30 12:41:58.508528 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:58.508501    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-5k947" event={"ID":"55710abf-0a0a-4830-a4dd-54083ab2ac04","Type":"ContainerStarted","Data":"1adb82688036cc1e51a3b02c0baf70844b545d800e020b02db037f2d29c21ae7"}
Oct 30 12:41:58.511273 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:41:58.511243    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9" event={"ID":"fed6ba8b-c3eb-444f-8def-0be29c763e81","Type":"ContainerStarted","Data":"6805d1ae96f9b8830012b109e007afd2c11d1277640332cc84814ee5e411e2fb"}
Oct 30 12:42:00.525901 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:00.525862    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-gzpff" event={"ID":"b818f62c-1645-4782-93a6-da2341cdcce7","Type":"ContainerStarted","Data":"a8c92614d73ad0869b0fa169ea7e251ac8534faf9d682ef1947941b000dbd1fd"}
Oct 30 12:42:00.526290 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:00.525998    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-gzpff"
Oct 30 12:42:00.570494 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:00.570413    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-gzpff" podStartSLOduration=1.174889765 podStartE2EDuration="3.570398012s" totalImagesPullingTime="2.395508247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:42:00.56730289 +0000 UTC m=+365.145251979" watchObservedRunningTime="2026-10-30 12:42:00.570398012 +0000 UTC m=+365.148347101"
Oct 30 12:42:01.532868 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:01.532815    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-n6rkn" event={"ID":"703d6675-c844-48eb-a019-2a09a2123307","Type":"ContainerStarted","Data":"7676f9a1db7ac96b9383adf8f8f466280d646805f6d20e146dd173c086a0c6b6"}
Oct 30 12:42:01.533314 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:01.533020    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-n6rkn"
Oct 30 12:42:01.534618 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:01.534573    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-5k947" event={"ID":"55710abf-0a0a-4830-a4dd-54083ab2ac04","Type":"ContainerStarted","Data":"265dfd03fba1519e98f41addbb035a56c782a10a689c40227294f06384d73023"}
Oct 30 12:42:01.534721 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:01.534668    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-5k947"
Oct 30 12:42:01.536275 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:01.536245    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9" event={"ID":"fed6ba8b-c3eb-444f-8def-0be29c763e81","Type":"ContainerStarted","Data":"f9270a53b200381121440ef9c33367deccb36f864892a92f1b0b0df9349705ee"}
Oct 30 12:42:01.536376 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:01.536318    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"
Oct 30 12:42:01.571972 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:01.571921    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-n6rkn" podStartSLOduration=1.260306098 podStartE2EDuration="4.57190777s" totalImagesPullingTime="3.311601672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:42:01.570931039 +0000 UTC m=+366.148880147" watchObservedRunningTime="2026-10-30 12:42:01.57190777 +0000 UTC m=+366.149856859"
Oct 30 12:42:01.611715 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:01.611649    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9" podStartSLOduration=1.731576248 podStartE2EDuration="4.611636388s" totalImagesPullingTime="2.88006014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:42:01.61130485 +0000 UTC m=+366.189253938" watchObservedRunningTime="2026-10-30 12:42:01.611636388 +0000 UTC m=+366.189585480"
Oct 30 12:42:01.653436 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:01.653386    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-5k947" podStartSLOduration=1.62818614 podStartE2EDuration="4.653371752s" totalImagesPullingTime="3.025185612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:41:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:42:01.650790287 +0000 UTC m=+366.228739389" watchObservedRunningTime="2026-10-30 12:42:01.653371752 +0000 UTC m=+366.231320838"
Oct 30 12:42:04.473092 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:04.473053    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-ktflf"
Oct 30 12:42:11.541080 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:11.541047    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-gzpff"
Oct 30 12:42:12.544568 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:12.544532    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-n6rkn"
Oct 30 12:42:21.593797 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:21.593751    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-wtc5x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:42:21.594284 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:21.593841    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x" podUID="8293650f-2035-41cb-8db9-7942342fd116" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:42:23.663063 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:23.662959    4945 generic.go:356] "Generic (PLEG): container finished" podID="8293650f-2035-41cb-8db9-7942342fd116" containerID="bd6c28a719533806985d76d39806e65acc3e2f42292f2897100320590bfeb490" exitCode=0
Oct 30 12:42:23.663063 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:23.663031    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x" event={"ID":"8293650f-2035-41cb-8db9-7942342fd116","Type":"ContainerDied","Data":"bd6c28a719533806985d76d39806e65acc3e2f42292f2897100320590bfeb490"}
Oct 30 12:42:23.663537 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:23.663451    4945 scope.go:122] "RemoveContainer" containerID="bd6c28a719533806985d76d39806e65acc3e2f42292f2897100320590bfeb490"
Oct 30 12:42:23.664549 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:23.664514    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:42:23.664670 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:23.664615    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:42:23.664734 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:23.664667    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:42:24.679638 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:24.670744    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x" event={"ID":"8293650f-2035-41cb-8db9-7942342fd116","Type":"ContainerStarted","Data":"235031e96c91f1a041fa65a912f5951f66cd9f913004f7e14316b523f02c3cb3"}
Oct 30 12:42:25.248003 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:25.247970    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:42:25.248274 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:25.248250    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-wtc5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Oct 30 12:42:25.248353 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:25.248300    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x" podUID="8293650f-2035-41cb-8db9-7942342fd116" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Oct 30 12:42:26.219279 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:42:26.219245    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:42:26.248738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:26.248706    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-wtc5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Oct 30 12:42:26.248888 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:26.248760    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x" podUID="8293650f-2035-41cb-8db9-7942342fd116" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Oct 30 12:42:27.248209 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:27.248174    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-wtc5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Oct 30 12:42:27.248586 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:27.248234    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x" podUID="8293650f-2035-41cb-8db9-7942342fd116" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Oct 30 12:42:28.247976 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:28.247937    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-wtc5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Oct 30 12:42:28.248146 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:28.247995    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x" podUID="8293650f-2035-41cb-8db9-7942342fd116" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Oct 30 12:42:29.248380 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:29.248344    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-wtc5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused" start-of-body=
Oct 30 12:42:29.248832 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:29.248395    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x" podUID="8293650f-2035-41cb-8db9-7942342fd116" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.29:15021/healthz/ready\": dial tcp 10.132.0.29:15021: connect: connection refused"
Oct 30 12:42:30.892896 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:30.892856    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:42:30.893413 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:30.893299    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:42:30.894830 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:30.894809    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 12:42:32.543738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:32.543708    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"
Oct 30 12:42:32.546698 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:32.546675    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-5k947"
Oct 30 12:42:33.567433 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.567394    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"]
Oct 30 12:42:33.620662 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.620623    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"]
Oct 30 12:42:33.620812 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.620719    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:33.674101 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.674067    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb36a502-be60-4114-9d7b-09cbdab505f5-openshift-service-ca\") pod \"maas-api-8664b8665c-7s9c8\" (UID: \"cb36a502-be60-4114-9d7b-09cbdab505f5\") " pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:33.674249 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.674117    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb36a502-be60-4114-9d7b-09cbdab505f5-metrics-certs\") pod \"maas-api-8664b8665c-7s9c8\" (UID: \"cb36a502-be60-4114-9d7b-09cbdab505f5\") " pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:33.674294 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.674268    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb36a502-be60-4114-9d7b-09cbdab505f5-maas-api-tls\") pod \"maas-api-8664b8665c-7s9c8\" (UID: \"cb36a502-be60-4114-9d7b-09cbdab505f5\") " pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:33.674330 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.674306    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltjz2\" (UniqueName: \"kubernetes.io/projected/cb36a502-be60-4114-9d7b-09cbdab505f5-kube-api-access-ltjz2\") pod \"maas-api-8664b8665c-7s9c8\" (UID: \"cb36a502-be60-4114-9d7b-09cbdab505f5\") " pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:33.687974 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.687948    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-mwj6w\""
Oct 30 12:42:33.688091 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.688025    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 12:42:33.688091 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.688063    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 12:42:33.775306 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.775275    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltjz2\" (UniqueName: \"kubernetes.io/projected/cb36a502-be60-4114-9d7b-09cbdab505f5-kube-api-access-ltjz2\") pod \"maas-api-8664b8665c-7s9c8\" (UID: \"cb36a502-be60-4114-9d7b-09cbdab505f5\") " pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:33.775511 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.775486    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb36a502-be60-4114-9d7b-09cbdab505f5-openshift-service-ca\") pod \"maas-api-8664b8665c-7s9c8\" (UID: \"cb36a502-be60-4114-9d7b-09cbdab505f5\") " pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:33.775642 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.775542    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb36a502-be60-4114-9d7b-09cbdab505f5-metrics-certs\") pod \"maas-api-8664b8665c-7s9c8\" (UID: \"cb36a502-be60-4114-9d7b-09cbdab505f5\") " pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:33.775698 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.775640    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb36a502-be60-4114-9d7b-09cbdab505f5-maas-api-tls\") pod \"maas-api-8664b8665c-7s9c8\" (UID: \"cb36a502-be60-4114-9d7b-09cbdab505f5\") " pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:33.776186 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.776165    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cb36a502-be60-4114-9d7b-09cbdab505f5-openshift-service-ca\") pod \"maas-api-8664b8665c-7s9c8\" (UID: \"cb36a502-be60-4114-9d7b-09cbdab505f5\") " pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:33.777993 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.777969    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cb36a502-be60-4114-9d7b-09cbdab505f5-metrics-certs\") pod \"maas-api-8664b8665c-7s9c8\" (UID: \"cb36a502-be60-4114-9d7b-09cbdab505f5\") " pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:33.778061 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.778018    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cb36a502-be60-4114-9d7b-09cbdab505f5-maas-api-tls\") pod \"maas-api-8664b8665c-7s9c8\" (UID: \"cb36a502-be60-4114-9d7b-09cbdab505f5\") " pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:33.796734 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.796705    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltjz2\" (UniqueName: \"kubernetes.io/projected/cb36a502-be60-4114-9d7b-09cbdab505f5-kube-api-access-ltjz2\") pod \"maas-api-8664b8665c-7s9c8\" (UID: \"cb36a502-be60-4114-9d7b-09cbdab505f5\") " pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:33.949062 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:33.949030    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:34.248080 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:34.248008    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"]
Oct 30 12:42:34.251433 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:42:34.251408    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb36a502_be60_4114_9d7b_09cbdab505f5.slice/crio-734b7de36910120ae3164dc010d4654760f86b9479c50b0e58233bc56d54846e WatchSource:0}: Error finding container 734b7de36910120ae3164dc010d4654760f86b9479c50b0e58233bc56d54846e: Status 404 returned error can't find the container with id 734b7de36910120ae3164dc010d4654760f86b9479c50b0e58233bc56d54846e
Oct 30 12:42:34.749158 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:34.749125    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8" event={"ID":"cb36a502-be60-4114-9d7b-09cbdab505f5","Type":"ContainerStarted","Data":"734b7de36910120ae3164dc010d4654760f86b9479c50b0e58233bc56d54846e"}
Oct 30 12:42:36.762835 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:36.762712    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8" event={"ID":"cb36a502-be60-4114-9d7b-09cbdab505f5","Type":"ContainerStarted","Data":"0186b8bb096efe139fd8b4dd1e1cb16d31ed28f6b3c5211e7561c8deb3ccc143"}
Oct 30 12:42:36.763306 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:36.762842    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:36.805251 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:36.805196    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8" podStartSLOduration=2.326442934 podStartE2EDuration="3.805182499s" totalImagesPullingTime="1.478739565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:42:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:42:36.802417636 +0000 UTC m=+401.380366724" watchObservedRunningTime="2026-10-30 12:42:36.805182499 +0000 UTC m=+401.383131587"
Oct 30 12:42:42.774522 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:42.774488    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8664b8665c-7s9c8"
Oct 30 12:42:56.190552 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:42:56.190518    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:42:57.298787 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:57.298747    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-z228w"]
Oct 30 12:42:57.302668 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:57.302643    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-z228w"
Oct 30 12:42:57.335326 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:57.335293    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-z228w"]
Oct 30 12:42:57.386805 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:57.386765    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fbcc3687-0c22-414a-aacc-5f18a80f7d41-cert\") pod \"llmisvc-controller-manager-6466c554d8-z228w\" (UID: \"fbcc3687-0c22-414a-aacc-5f18a80f7d41\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-z228w"
Oct 30 12:42:57.386934 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:57.386882    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw2l2\" (UniqueName: \"kubernetes.io/projected/fbcc3687-0c22-414a-aacc-5f18a80f7d41-kube-api-access-fw2l2\") pod \"llmisvc-controller-manager-6466c554d8-z228w\" (UID: \"fbcc3687-0c22-414a-aacc-5f18a80f7d41\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-z228w"
Oct 30 12:42:57.487984 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:57.487960    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw2l2\" (UniqueName: \"kubernetes.io/projected/fbcc3687-0c22-414a-aacc-5f18a80f7d41-kube-api-access-fw2l2\") pod \"llmisvc-controller-manager-6466c554d8-z228w\" (UID: \"fbcc3687-0c22-414a-aacc-5f18a80f7d41\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-z228w"
Oct 30 12:42:57.488114 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:57.488020    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fbcc3687-0c22-414a-aacc-5f18a80f7d41-cert\") pod \"llmisvc-controller-manager-6466c554d8-z228w\" (UID: \"fbcc3687-0c22-414a-aacc-5f18a80f7d41\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-z228w"
Oct 30 12:42:57.490323 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:57.490298    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fbcc3687-0c22-414a-aacc-5f18a80f7d41-cert\") pod \"llmisvc-controller-manager-6466c554d8-z228w\" (UID: \"fbcc3687-0c22-414a-aacc-5f18a80f7d41\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-z228w"
Oct 30 12:42:57.506822 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:57.506798    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw2l2\" (UniqueName: \"kubernetes.io/projected/fbcc3687-0c22-414a-aacc-5f18a80f7d41-kube-api-access-fw2l2\") pod \"llmisvc-controller-manager-6466c554d8-z228w\" (UID: \"fbcc3687-0c22-414a-aacc-5f18a80f7d41\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-z228w"
Oct 30 12:42:57.615090 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:57.615025    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-z228w"
Oct 30 12:42:57.907413 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:57.907257    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-z228w"]
Oct 30 12:42:57.910235 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:42:57.910203    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbcc3687_0c22_414a_aacc_5f18a80f7d41.slice/crio-297e0c164d3b0e6166216e44aa8ccb07741a46e7d434017402bd4c654861e548 WatchSource:0}: Error finding container 297e0c164d3b0e6166216e44aa8ccb07741a46e7d434017402bd4c654861e548: Status 404 returned error can't find the container with id 297e0c164d3b0e6166216e44aa8ccb07741a46e7d434017402bd4c654861e548
Oct 30 12:42:58.878689 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:58.878648    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-z228w" event={"ID":"fbcc3687-0c22-414a-aacc-5f18a80f7d41","Type":"ContainerStarted","Data":"8ae44479e842f580a2c26ee172b897638140c6251a99d0b66fb9c732eb1d6897"}
Oct 30 12:42:58.879107 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:58.878733    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-z228w" event={"ID":"fbcc3687-0c22-414a-aacc-5f18a80f7d41","Type":"ContainerStarted","Data":"297e0c164d3b0e6166216e44aa8ccb07741a46e7d434017402bd4c654861e548"}
Oct 30 12:42:58.879107 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:58.878888    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-z228w"
Oct 30 12:42:58.916160 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:42:58.916109    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-z228w" podStartSLOduration=1.65366226 podStartE2EDuration="1.91609733s" totalImagesPullingTime="262.43507ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:42:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:42:58.914950779 +0000 UTC m=+423.492899869" watchObservedRunningTime="2026-10-30 12:42:58.91609733 +0000 UTC m=+423.494046420"
Oct 30 12:43:26.190288 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:43:26.190242    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:43:29.886526 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:29.886497    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-z228w"
Oct 30 12:43:29.956429 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:29.956377    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"]
Oct 30 12:43:29.956728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:29.956697    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9" podUID="fed6ba8b-c3eb-444f-8def-0be29c763e81" containerName="manager" containerID="cri-o://f9270a53b200381121440ef9c33367deccb36f864892a92f1b0b0df9349705ee" gracePeriod=30
Oct 30 12:43:30.212948 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:30.212924    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"
Oct 30 12:43:30.270084 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:30.270051    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fed6ba8b-c3eb-444f-8def-0be29c763e81-kube-api-access-6zbwc\" (UniqueName: \"kubernetes.io/projected/fed6ba8b-c3eb-444f-8def-0be29c763e81-kube-api-access-6zbwc\") pod \"fed6ba8b-c3eb-444f-8def-0be29c763e81\" (UID: \"fed6ba8b-c3eb-444f-8def-0be29c763e81\") "
Oct 30 12:43:30.270276 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:30.270117    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fed6ba8b-c3eb-444f-8def-0be29c763e81-cert\" (UniqueName: \"kubernetes.io/secret/fed6ba8b-c3eb-444f-8def-0be29c763e81-cert\") pod \"fed6ba8b-c3eb-444f-8def-0be29c763e81\" (UID: \"fed6ba8b-c3eb-444f-8def-0be29c763e81\") "
Oct 30 12:43:30.272260 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:30.272229    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fed6ba8b-c3eb-444f-8def-0be29c763e81-cert" pod "fed6ba8b-c3eb-444f-8def-0be29c763e81" (UID: "fed6ba8b-c3eb-444f-8def-0be29c763e81"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:43:30.272260 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:30.272254    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fed6ba8b-c3eb-444f-8def-0be29c763e81-kube-api-access-6zbwc" pod "fed6ba8b-c3eb-444f-8def-0be29c763e81" (UID: "fed6ba8b-c3eb-444f-8def-0be29c763e81"). InnerVolumeSpecName "kube-api-access-6zbwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:43:30.370932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:30.370904    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zbwc\" (UniqueName: \"kubernetes.io/projected/fed6ba8b-c3eb-444f-8def-0be29c763e81-kube-api-access-6zbwc\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:43:30.370932 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:30.370931    4945 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fed6ba8b-c3eb-444f-8def-0be29c763e81-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:43:31.046153 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:31.046118    4945 generic.go:356] "Generic (PLEG): container finished" podID="fed6ba8b-c3eb-444f-8def-0be29c763e81" containerID="f9270a53b200381121440ef9c33367deccb36f864892a92f1b0b0df9349705ee" exitCode=0
Oct 30 12:43:31.046687 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:31.046178    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"
Oct 30 12:43:31.046687 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:31.046214    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9" event={"ID":"fed6ba8b-c3eb-444f-8def-0be29c763e81","Type":"ContainerDied","Data":"f9270a53b200381121440ef9c33367deccb36f864892a92f1b0b0df9349705ee"}
Oct 30 12:43:31.046687 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:31.046270    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9" event={"ID":"fed6ba8b-c3eb-444f-8def-0be29c763e81","Type":"ContainerDied","Data":"6805d1ae96f9b8830012b109e007afd2c11d1277640332cc84814ee5e411e2fb"}
Oct 30 12:43:31.046687 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:31.046293    4945 scope.go:122] "RemoveContainer" containerID="f9270a53b200381121440ef9c33367deccb36f864892a92f1b0b0df9349705ee"
Oct 30 12:43:31.056281 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:31.056265    4945 scope.go:122] "RemoveContainer" containerID="f9270a53b200381121440ef9c33367deccb36f864892a92f1b0b0df9349705ee"
Oct 30 12:43:31.056563 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:43:31.056544    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9270a53b200381121440ef9c33367deccb36f864892a92f1b0b0df9349705ee\": container with ID starting with f9270a53b200381121440ef9c33367deccb36f864892a92f1b0b0df9349705ee not found: ID does not exist" containerID="f9270a53b200381121440ef9c33367deccb36f864892a92f1b0b0df9349705ee"
Oct 30 12:43:31.056646 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:31.056573    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9270a53b200381121440ef9c33367deccb36f864892a92f1b0b0df9349705ee"} err="failed to get container status \"f9270a53b200381121440ef9c33367deccb36f864892a92f1b0b0df9349705ee\": rpc error: code = NotFound desc = could not find container \"f9270a53b200381121440ef9c33367deccb36f864892a92f1b0b0df9349705ee\": container with ID starting with f9270a53b200381121440ef9c33367deccb36f864892a92f1b0b0df9349705ee not found: ID does not exist"
Oct 30 12:43:31.095620 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:31.095575    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"]
Oct 30 12:43:31.106810 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:31.106786    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-xvtl9"]
Oct 30 12:43:31.547946 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:43:31.547915    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fed6ba8b-c3eb-444f-8def-0be29c763e81" path="/var/lib/kubelet/pods/fed6ba8b-c3eb-444f-8def-0be29c763e81/volumes"
Oct 30 12:43:56.193540 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:43:56.193503    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:44:10.260640 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:44:10.260582    4945 generic.go:356] "Generic (PLEG): container finished" podID="180c4327-13ad-48b0-b5f1-c193ab9b0c9d" containerID="eabc9f9ffee979a4a8ac0527a248557cbd7884ec9c0638379ba5e4794d3c50fa" exitCode=1
Oct 30 12:44:10.261089 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:44:10.260656    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d7d58c6ff-pncqs" event={"ID":"180c4327-13ad-48b0-b5f1-c193ab9b0c9d","Type":"ContainerDied","Data":"eabc9f9ffee979a4a8ac0527a248557cbd7884ec9c0638379ba5e4794d3c50fa"}
Oct 30 12:44:10.261089 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:44:10.261020    4945 scope.go:122] "RemoveContainer" containerID="eabc9f9ffee979a4a8ac0527a248557cbd7884ec9c0638379ba5e4794d3c50fa"
Oct 30 12:44:11.268500 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:44:11.268464    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d7d58c6ff-pncqs" event={"ID":"180c4327-13ad-48b0-b5f1-c193ab9b0c9d","Type":"ContainerStarted","Data":"d46adb25d117eb468c6e54a60be7d32448f2712e01d1043c2127ff406589ae32"}
Oct 30 12:44:11.268897 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:44:11.268745    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5d7d58c6ff-pncqs"
Oct 30 12:44:22.276454 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:44:22.276421    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5d7d58c6ff-pncqs"
Oct 30 12:44:26.190808 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:44:26.190775    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:44:56.192020 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:44:56.191982    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:45:13.907586 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:13.907544    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"]
Oct 30 12:45:13.908215 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:13.908197    4945 cpu_manager.go:414] "containerMap: removing container" podUID="fed6ba8b-c3eb-444f-8def-0be29c763e81" containerName="manager"
Oct 30 12:45:13.908295 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:13.908218    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fed6ba8b-c3eb-444f-8def-0be29c763e81" containerName="manager"
Oct 30 12:45:13.908400 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:13.908388    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="fed6ba8b-c3eb-444f-8def-0be29c763e81" containerName="manager"
Oct 30 12:45:13.912112 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:13.912090    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:13.942574 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:13.942548    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"]
Oct 30 12:45:14.008305 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.008266    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.008305 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.008308    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf485\" (UniqueName: \"kubernetes.io/projected/951278f9-b9f4-43bd-920c-fb92e3fbed25-kube-api-access-hf485\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.008546 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.008332    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.008546 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.008476    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.008546 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.008519    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/951278f9-b9f4-43bd-920c-fb92e3fbed25-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.008784 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.008671    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.008784 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.008701    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.008784 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.008762    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.008937 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.008847    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.109190 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.109162    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.109190 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.109191    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf485\" (UniqueName: \"kubernetes.io/projected/951278f9-b9f4-43bd-920c-fb92e3fbed25-kube-api-access-hf485\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.109416 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.109215    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.109416 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.109242    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.109416 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.109268    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/951278f9-b9f4-43bd-920c-fb92e3fbed25-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.109416 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.109328    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.109416 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.109381    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.109690 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.109418    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.109690 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.109467    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.109690 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.109634    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.109970 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.109936    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.110105 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.109954    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/951278f9-b9f4-43bd-920c-fb92e3fbed25-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.110105 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.110090    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.110242 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.110219    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.112046 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.112018    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.112149 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.112092    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.127648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.127624    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.128187 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.128169    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf485\" (UniqueName: \"kubernetes.io/projected/951278f9-b9f4-43bd-920c-fb92e3fbed25-kube-api-access-hf485\") pod \"maas-default-gateway-openshift-default-7d9897b54c-f74x8\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.227838 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.227811    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:14.538933 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.538850    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"]
Oct 30 12:45:14.541411 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:45:14.541373    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod951278f9_b9f4_43bd_920c_fb92e3fbed25.slice/crio-e87ed63ba0b9a3766bc7383658b76565d5147fd3a839fa8024e4a48de2a05676 WatchSource:0}: Error finding container e87ed63ba0b9a3766bc7383658b76565d5147fd3a839fa8024e4a48de2a05676: Status 404 returned error can't find the container with id e87ed63ba0b9a3766bc7383658b76565d5147fd3a839fa8024e4a48de2a05676
Oct 30 12:45:14.543533 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.543504    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:45:14.543628 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.543564    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:45:14.543628 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.543592    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:45:14.646317 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.646283    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8" event={"ID":"951278f9-b9f4-43bd-920c-fb92e3fbed25","Type":"ContainerStarted","Data":"146062a33cfe6066a21f08340a14679945e26f6d406497d0689f6170d39f02b2"}
Oct 30 12:45:14.646470 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.646345    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8" event={"ID":"951278f9-b9f4-43bd-920c-fb92e3fbed25","Type":"ContainerStarted","Data":"e87ed63ba0b9a3766bc7383658b76565d5147fd3a839fa8024e4a48de2a05676"}
Oct 30 12:45:14.689536 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:14.689475    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8" podStartSLOduration=1.6894555329999998 podStartE2EDuration="1.689455533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:45:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:45:14.686235186 +0000 UTC m=+559.264184259" watchObservedRunningTime="2026-10-30 12:45:14.689455533 +0000 UTC m=+559.267404623"
Oct 30 12:45:15.228274 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:15.228220    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:15.230031 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:15.230002    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-f74x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 12:45:15.230164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:15.230050    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8" podUID="951278f9-b9f4-43bd-920c-fb92e3fbed25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 12:45:16.228588 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:16.228545    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-f74x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 12:45:16.228996 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:16.228708    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8" podUID="951278f9-b9f4-43bd-920c-fb92e3fbed25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 12:45:17.228454 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:17.228413    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-f74x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 12:45:17.228677 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:17.228472    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8" podUID="951278f9-b9f4-43bd-920c-fb92e3fbed25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 12:45:18.228877 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:18.228839    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-f74x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 12:45:18.229314 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:18.228895    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8" podUID="951278f9-b9f4-43bd-920c-fb92e3fbed25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 12:45:19.229146 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:19.229102    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-f74x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 12:45:19.229673 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:19.229182    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8" podUID="951278f9-b9f4-43bd-920c-fb92e3fbed25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 12:45:20.229147 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:20.229105    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-f74x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 12:45:20.229332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:20.229189    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8" podUID="951278f9-b9f4-43bd-920c-fb92e3fbed25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 12:45:21.228841 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:21.228803    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-f74x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 12:45:21.229013 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:21.228876    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8" podUID="951278f9-b9f4-43bd-920c-fb92e3fbed25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 12:45:22.927634 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:22.927590    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:22.928086 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:22.927985    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:22.928701 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:22.928681    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 12:45:26.189974 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:45:26.189936    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:45:28.983974 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:28.983941    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"]
Oct 30 12:45:28.987878 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:28.987862    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.024737 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.024707    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"]
Oct 30 12:45:29.058934 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.058910    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8885359-7cb6-4e2e-89ca-c1499f813ea7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.059071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.058952    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8885359-7cb6-4e2e-89ca-c1499f813ea7-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.059071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.058975    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qhgv\" (UniqueName: \"kubernetes.io/projected/c8885359-7cb6-4e2e-89ca-c1499f813ea7-kube-api-access-4qhgv\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.059190 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.059102    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8885359-7cb6-4e2e-89ca-c1499f813ea7-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.059190 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.059139    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8885359-7cb6-4e2e-89ca-c1499f813ea7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.059271 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.059207    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8885359-7cb6-4e2e-89ca-c1499f813ea7-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.059313 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.059278    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8885359-7cb6-4e2e-89ca-c1499f813ea7-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.059367 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.059313    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8885359-7cb6-4e2e-89ca-c1499f813ea7-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.059429 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.059413    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8885359-7cb6-4e2e-89ca-c1499f813ea7-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.160795 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.160760    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8885359-7cb6-4e2e-89ca-c1499f813ea7-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.160944 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.160845    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8885359-7cb6-4e2e-89ca-c1499f813ea7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.160944 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.160880    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8885359-7cb6-4e2e-89ca-c1499f813ea7-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.160944 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.160900    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qhgv\" (UniqueName: \"kubernetes.io/projected/c8885359-7cb6-4e2e-89ca-c1499f813ea7-kube-api-access-4qhgv\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.160944 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.160921    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8885359-7cb6-4e2e-89ca-c1499f813ea7-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.161142 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.160945    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8885359-7cb6-4e2e-89ca-c1499f813ea7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.161142 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.160976    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8885359-7cb6-4e2e-89ca-c1499f813ea7-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.161142 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.161016    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8885359-7cb6-4e2e-89ca-c1499f813ea7-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.161142 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.161047    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8885359-7cb6-4e2e-89ca-c1499f813ea7-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.161363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.161337    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8885359-7cb6-4e2e-89ca-c1499f813ea7-istio-data\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.161423 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.161362    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8885359-7cb6-4e2e-89ca-c1499f813ea7-credential-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.161559 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.161494    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8885359-7cb6-4e2e-89ca-c1499f813ea7-workload-socket\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.161559 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.161513    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8885359-7cb6-4e2e-89ca-c1499f813ea7-workload-certs\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.161811 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.161788    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8885359-7cb6-4e2e-89ca-c1499f813ea7-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.163147 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.163122    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8885359-7cb6-4e2e-89ca-c1499f813ea7-istio-envoy\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.163391 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.163369    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8885359-7cb6-4e2e-89ca-c1499f813ea7-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.180048 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.180028    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8885359-7cb6-4e2e-89ca-c1499f813ea7-istio-token\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.182506 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.182485    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qhgv\" (UniqueName: \"kubernetes.io/projected/c8885359-7cb6-4e2e-89ca-c1499f813ea7-kube-api-access-4qhgv\") pod \"maas-default-gateway-openshift-default-7d9897b54c-5fwdk\" (UID: \"c8885359-7cb6-4e2e-89ca-c1499f813ea7\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.301576 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.301501    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:29.613120 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.613006    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"]
Oct 30 12:45:29.615859 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:45:29.615832    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8885359_7cb6_4e2e_89ca_c1499f813ea7.slice/crio-92e63bb9618badab90df5e44d86b8448f1865961328f8c09aff44f86e89db77b WatchSource:0}: Error finding container 92e63bb9618badab90df5e44d86b8448f1865961328f8c09aff44f86e89db77b: Status 404 returned error can't find the container with id 92e63bb9618badab90df5e44d86b8448f1865961328f8c09aff44f86e89db77b
Oct 30 12:45:29.618080 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.618049    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:45:29.618156 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.618116    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:45:29.618201 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.618172    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:45:29.738341 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.738305    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk" event={"ID":"c8885359-7cb6-4e2e-89ca-c1499f813ea7","Type":"ContainerStarted","Data":"c65b1b364ab1944d9c667fa597ec0a4a7090aaca02fe761a5a308696a8bc20a5"}
Oct 30 12:45:29.738510 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.738361    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk" event={"ID":"c8885359-7cb6-4e2e-89ca-c1499f813ea7","Type":"ContainerStarted","Data":"92e63bb9618badab90df5e44d86b8448f1865961328f8c09aff44f86e89db77b"}
Oct 30 12:45:29.781024 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:29.780978    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk" podStartSLOduration=1.780964059 podStartE2EDuration="1.780964059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:45:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:45:29.778359165 +0000 UTC m=+574.356308253" watchObservedRunningTime="2026-10-30 12:45:29.780964059 +0000 UTC m=+574.358913147"
Oct 30 12:45:30.302796 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:30.302758    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:30.304301 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:30.304278    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-5fwdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 12:45:30.304370 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:30.304328    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk" podUID="c8885359-7cb6-4e2e-89ca-c1499f813ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 12:45:31.302692 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:31.302656    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-5fwdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 12:45:31.302880 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:31.302725    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk" podUID="c8885359-7cb6-4e2e-89ca-c1499f813ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 12:45:32.302172 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:32.302138    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-5fwdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 12:45:32.302355 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:32.302196    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk" podUID="c8885359-7cb6-4e2e-89ca-c1499f813ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 12:45:33.302120 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:33.302083    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-5fwdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 12:45:33.302529 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:33.302144    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk" podUID="c8885359-7cb6-4e2e-89ca-c1499f813ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 12:45:34.302264 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:34.302231    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-5fwdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 12:45:34.302672 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:34.302282    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk" podUID="c8885359-7cb6-4e2e-89ca-c1499f813ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 12:45:35.302841 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:35.302799    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-5fwdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 12:45:35.303341 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:35.302874    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk" podUID="c8885359-7cb6-4e2e-89ca-c1499f813ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 12:45:36.302042 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:36.302006    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-5fwdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Oct 30 12:45:36.302305 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:36.302077    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk" podUID="c8885359-7cb6-4e2e-89ca-c1499f813ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Oct 30 12:45:38.001931 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:38.001900    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:38.002624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:38.002555    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:38.003368 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:38.003348    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-5fwdk"
Oct 30 12:45:55.626049 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:55.626018    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 12:45:55.626542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:45:55.626262    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 12:45:56.190495 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:45:56.190450    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:46:09.914734 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:09.914695    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"]
Oct 30 12:46:09.918669 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:09.918641    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:09.953721 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:09.953580    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"]
Oct 30 12:46:09.960148 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:09.960109    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5431fa34-serving-cert\""
Oct 30 12:46:10.060752 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.060717    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxfdp\" (UniqueName: \"kubernetes.io/projected/0e6959df-26a0-4340-bc8d-4add6a0a798f-kube-api-access-xxfdp\") pod \"maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:10.060936 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.060775    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e6959df-26a0-4340-bc8d-4add6a0a798f-maas-api-tls\") pod \"maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:10.060936 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.060843    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e6959df-26a0-4340-bc8d-4add6a0a798f-metrics-certs\") pod \"maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:10.061056 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.060975    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e6959df-26a0-4340-bc8d-4add6a0a798f-openshift-service-ca\") pod \"maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:10.161723 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.161675    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e6959df-26a0-4340-bc8d-4add6a0a798f-openshift-service-ca\") pod \"maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:10.161920 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.161832    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxfdp\" (UniqueName: \"kubernetes.io/projected/0e6959df-26a0-4340-bc8d-4add6a0a798f-kube-api-access-xxfdp\") pod \"maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:10.161920 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.161883    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e6959df-26a0-4340-bc8d-4add6a0a798f-maas-api-tls\") pod \"maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:10.161920 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.161900    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e6959df-26a0-4340-bc8d-4add6a0a798f-metrics-certs\") pod \"maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:10.162576 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.162544    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e6959df-26a0-4340-bc8d-4add6a0a798f-openshift-service-ca\") pod \"maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:10.165104 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.165047    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e6959df-26a0-4340-bc8d-4add6a0a798f-maas-api-tls\") pod \"maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:10.167290 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.167268    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e6959df-26a0-4340-bc8d-4add6a0a798f-metrics-certs\") pod \"maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:10.185659 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.185629    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxfdp\" (UniqueName: \"kubernetes.io/projected/0e6959df-26a0-4340-bc8d-4add6a0a798f-kube-api-access-xxfdp\") pod \"maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:10.231549 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.231512    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:10.540051 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.540026    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"]
Oct 30 12:46:10.542570 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:46:10.542544    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e6959df_26a0_4340_bc8d_4add6a0a798f.slice/crio-f37a33108d4dff42b0ccc071552cce1f8a4603b99186631fd215928748bed23c WatchSource:0}: Error finding container f37a33108d4dff42b0ccc071552cce1f8a4603b99186631fd215928748bed23c: Status 404 returned error can't find the container with id f37a33108d4dff42b0ccc071552cce1f8a4603b99186631fd215928748bed23c
Oct 30 12:46:10.983224 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.983170    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq" event={"ID":"0e6959df-26a0-4340-bc8d-4add6a0a798f","Type":"ContainerStarted","Data":"fe4d1668773ab5fdbfc2d3867590863ca361cf6f2c85883ccb5b197393bfc992"}
Oct 30 12:46:10.983749 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.983257    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq" event={"ID":"0e6959df-26a0-4340-bc8d-4add6a0a798f","Type":"ContainerStarted","Data":"f37a33108d4dff42b0ccc071552cce1f8a4603b99186631fd215928748bed23c"}
Oct 30 12:46:10.984010 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:10.983989    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:11.028840 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:11.028753    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq" podStartSLOduration=1.708145818 podStartE2EDuration="2.028738614s" totalImagesPullingTime="320.592796ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:46:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:46:11.028363817 +0000 UTC m=+615.606312905" watchObservedRunningTime="2026-10-30 12:46:11.028738614 +0000 UTC m=+615.606687703"
Oct 30 12:46:18.002664 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:18.002627    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:21.487742 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.487700    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"]
Oct 30 12:46:21.491926 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.491900    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:21.525741 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.525713    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-48623dd7-serving-cert\""
Oct 30 12:46:21.526651 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.526625    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"]
Oct 30 12:46:21.576956 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.576922    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4dbe9b-0326-4112-b64f-003ba69ac14f-openshift-service-ca\") pod \"maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:21.577150 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.577039    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kps68\" (UniqueName: \"kubernetes.io/projected/1f4dbe9b-0326-4112-b64f-003ba69ac14f-kube-api-access-kps68\") pod \"maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:21.577150 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.577117    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f4dbe9b-0326-4112-b64f-003ba69ac14f-metrics-certs\") pod \"maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:21.577231 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.577208    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f4dbe9b-0326-4112-b64f-003ba69ac14f-maas-api-tls\") pod \"maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:21.678479 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.678445    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4dbe9b-0326-4112-b64f-003ba69ac14f-openshift-service-ca\") pod \"maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:21.678479 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.678482    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kps68\" (UniqueName: \"kubernetes.io/projected/1f4dbe9b-0326-4112-b64f-003ba69ac14f-kube-api-access-kps68\") pod \"maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:21.678714 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.678502    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f4dbe9b-0326-4112-b64f-003ba69ac14f-metrics-certs\") pod \"maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:21.678714 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.678534    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f4dbe9b-0326-4112-b64f-003ba69ac14f-maas-api-tls\") pod \"maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:21.679163 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.679133    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4dbe9b-0326-4112-b64f-003ba69ac14f-openshift-service-ca\") pod \"maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:21.681007 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.680982    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f4dbe9b-0326-4112-b64f-003ba69ac14f-metrics-certs\") pod \"maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:21.681135 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.681078    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f4dbe9b-0326-4112-b64f-003ba69ac14f-maas-api-tls\") pod \"maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:21.697432 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.697409    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kps68\" (UniqueName: \"kubernetes.io/projected/1f4dbe9b-0326-4112-b64f-003ba69ac14f-kube-api-access-kps68\") pod \"maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:21.805849 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:21.805753    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:22.106159 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:22.106052    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"]
Oct 30 12:46:22.108988 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:46:22.108962    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f4dbe9b_0326_4112_b64f_003ba69ac14f.slice/crio-65a7dfde05cc19abe6fe2e94b31848517513305b1580280a1ba5432b67afe629 WatchSource:0}: Error finding container 65a7dfde05cc19abe6fe2e94b31848517513305b1580280a1ba5432b67afe629: Status 404 returned error can't find the container with id 65a7dfde05cc19abe6fe2e94b31848517513305b1580280a1ba5432b67afe629
Oct 30 12:46:22.110275 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:22.110257    4945 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 12:46:23.053994 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:23.053959    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f" event={"ID":"1f4dbe9b-0326-4112-b64f-003ba69ac14f","Type":"ContainerStarted","Data":"fe9eab2cc08a1fd541d8f8c7b46d9b39dec2923c9e055be9f6c64df73b909c62"}
Oct 30 12:46:23.054504 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:23.054016    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f" event={"ID":"1f4dbe9b-0326-4112-b64f-003ba69ac14f","Type":"ContainerStarted","Data":"65a7dfde05cc19abe6fe2e94b31848517513305b1580280a1ba5432b67afe629"}
Oct 30 12:46:23.054504 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:23.054042    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:23.094608 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:23.094548    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f" podStartSLOduration=1.7230808930000001 podStartE2EDuration="2.094529544s" totalImagesPullingTime="371.448651ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:46:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:46:23.092100697 +0000 UTC m=+627.670049787" watchObservedRunningTime="2026-10-30 12:46:23.094529544 +0000 UTC m=+627.672478635"
Oct 30 12:46:26.192168 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:46:26.192126    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:46:29.064556 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:29.064528    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:39.158755 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:39.158713    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"]
Oct 30 12:46:39.159221 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:39.158997    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq" podUID="0e6959df-26a0-4340-bc8d-4add6a0a798f" containerName="maas-api" containerID="cri-o://fe4d1668773ab5fdbfc2d3867590863ca361cf6f2c85883ccb5b197393bfc992" gracePeriod=30
Oct 30 12:46:39.458655 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:39.458572    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:39.577868 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:39.577832    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e6959df-26a0-4340-bc8d-4add6a0a798f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e6959df-26a0-4340-bc8d-4add6a0a798f-maas-api-tls\") pod \"0e6959df-26a0-4340-bc8d-4add6a0a798f\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") "
Oct 30 12:46:39.578029 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:39.577918    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e6959df-26a0-4340-bc8d-4add6a0a798f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e6959df-26a0-4340-bc8d-4add6a0a798f-metrics-certs\") pod \"0e6959df-26a0-4340-bc8d-4add6a0a798f\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") "
Oct 30 12:46:39.578029 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:39.577969    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e6959df-26a0-4340-bc8d-4add6a0a798f-kube-api-access-xxfdp\" (UniqueName: \"kubernetes.io/projected/0e6959df-26a0-4340-bc8d-4add6a0a798f-kube-api-access-xxfdp\") pod \"0e6959df-26a0-4340-bc8d-4add6a0a798f\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") "
Oct 30 12:46:39.578117 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:39.578039    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e6959df-26a0-4340-bc8d-4add6a0a798f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e6959df-26a0-4340-bc8d-4add6a0a798f-openshift-service-ca\") pod \"0e6959df-26a0-4340-bc8d-4add6a0a798f\" (UID: \"0e6959df-26a0-4340-bc8d-4add6a0a798f\") "
Oct 30 12:46:39.578492 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:39.578450    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e6959df-26a0-4340-bc8d-4add6a0a798f-openshift-service-ca" pod "0e6959df-26a0-4340-bc8d-4add6a0a798f" (UID: "0e6959df-26a0-4340-bc8d-4add6a0a798f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:46:39.580216 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:39.580182    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e6959df-26a0-4340-bc8d-4add6a0a798f-kube-api-access-xxfdp" pod "0e6959df-26a0-4340-bc8d-4add6a0a798f" (UID: "0e6959df-26a0-4340-bc8d-4add6a0a798f"). InnerVolumeSpecName "kube-api-access-xxfdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:46:39.580329 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:39.580212    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e6959df-26a0-4340-bc8d-4add6a0a798f-maas-api-tls" pod "0e6959df-26a0-4340-bc8d-4add6a0a798f" (UID: "0e6959df-26a0-4340-bc8d-4add6a0a798f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:46:39.580329 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:39.580226    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e6959df-26a0-4340-bc8d-4add6a0a798f-metrics-certs" pod "0e6959df-26a0-4340-bc8d-4add6a0a798f" (UID: "0e6959df-26a0-4340-bc8d-4add6a0a798f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:46:39.681642 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:39.679910    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxfdp\" (UniqueName: \"kubernetes.io/projected/0e6959df-26a0-4340-bc8d-4add6a0a798f-kube-api-access-xxfdp\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:46:39.681642 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:39.679949    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e6959df-26a0-4340-bc8d-4add6a0a798f-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:46:39.681642 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:39.679964    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e6959df-26a0-4340-bc8d-4add6a0a798f-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:46:39.681642 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:39.679976    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e6959df-26a0-4340-bc8d-4add6a0a798f-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:46:40.159589 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:40.159557    4945 generic.go:356] "Generic (PLEG): container finished" podID="0e6959df-26a0-4340-bc8d-4add6a0a798f" containerID="fe4d1668773ab5fdbfc2d3867590863ca361cf6f2c85883ccb5b197393bfc992" exitCode=0
Oct 30 12:46:40.159983 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:40.159642    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"
Oct 30 12:46:40.159983 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:40.159637    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq" event={"ID":"0e6959df-26a0-4340-bc8d-4add6a0a798f","Type":"ContainerDied","Data":"fe4d1668773ab5fdbfc2d3867590863ca361cf6f2c85883ccb5b197393bfc992"}
Oct 30 12:46:40.159983 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:40.159767    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq" event={"ID":"0e6959df-26a0-4340-bc8d-4add6a0a798f","Type":"ContainerDied","Data":"f37a33108d4dff42b0ccc071552cce1f8a4603b99186631fd215928748bed23c"}
Oct 30 12:46:40.159983 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:40.159791    4945 scope.go:122] "RemoveContainer" containerID="fe4d1668773ab5fdbfc2d3867590863ca361cf6f2c85883ccb5b197393bfc992"
Oct 30 12:46:40.172081 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:40.172058    4945 scope.go:122] "RemoveContainer" containerID="fe4d1668773ab5fdbfc2d3867590863ca361cf6f2c85883ccb5b197393bfc992"
Oct 30 12:46:40.172485 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:46:40.172456    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe4d1668773ab5fdbfc2d3867590863ca361cf6f2c85883ccb5b197393bfc992\": container with ID starting with fe4d1668773ab5fdbfc2d3867590863ca361cf6f2c85883ccb5b197393bfc992 not found: ID does not exist" containerID="fe4d1668773ab5fdbfc2d3867590863ca361cf6f2c85883ccb5b197393bfc992"
Oct 30 12:46:40.172577 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:40.172489    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe4d1668773ab5fdbfc2d3867590863ca361cf6f2c85883ccb5b197393bfc992"} err="failed to get container status \"fe4d1668773ab5fdbfc2d3867590863ca361cf6f2c85883ccb5b197393bfc992\": rpc error: code = NotFound desc = could not find container \"fe4d1668773ab5fdbfc2d3867590863ca361cf6f2c85883ccb5b197393bfc992\": container with ID starting with fe4d1668773ab5fdbfc2d3867590863ca361cf6f2c85883ccb5b197393bfc992 not found: ID does not exist"
Oct 30 12:46:40.209278 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:40.209246    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"]
Oct 30 12:46:40.221180 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:40.221154    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5431fa34-8cf967dd9-rrqzq"]
Oct 30 12:46:41.548941 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:41.548909    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e6959df-26a0-4340-bc8d-4add6a0a798f" path="/var/lib/kubelet/pods/0e6959df-26a0-4340-bc8d-4add6a0a798f/volumes"
Oct 30 12:46:50.157803 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:50.157767    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"]
Oct 30 12:46:50.158638 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:50.158549    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f" podUID="1f4dbe9b-0326-4112-b64f-003ba69ac14f" containerName="maas-api" containerID="cri-o://fe9eab2cc08a1fd541d8f8c7b46d9b39dec2923c9e055be9f6c64df73b909c62" gracePeriod=30
Oct 30 12:46:50.425537 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:50.425510    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:50.617581 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:50.616283    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f4dbe9b-0326-4112-b64f-003ba69ac14f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4dbe9b-0326-4112-b64f-003ba69ac14f-openshift-service-ca\") pod \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") "
Oct 30 12:46:50.617581 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:50.616364    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f4dbe9b-0326-4112-b64f-003ba69ac14f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f4dbe9b-0326-4112-b64f-003ba69ac14f-maas-api-tls\") pod \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") "
Oct 30 12:46:50.617581 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:50.616471    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f4dbe9b-0326-4112-b64f-003ba69ac14f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f4dbe9b-0326-4112-b64f-003ba69ac14f-metrics-certs\") pod \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") "
Oct 30 12:46:50.617581 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:50.616507    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f4dbe9b-0326-4112-b64f-003ba69ac14f-kube-api-access-kps68\" (UniqueName: \"kubernetes.io/projected/1f4dbe9b-0326-4112-b64f-003ba69ac14f-kube-api-access-kps68\") pod \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\" (UID: \"1f4dbe9b-0326-4112-b64f-003ba69ac14f\") "
Oct 30 12:46:50.619162 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:50.619119    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f4dbe9b-0326-4112-b64f-003ba69ac14f-openshift-service-ca" pod "1f4dbe9b-0326-4112-b64f-003ba69ac14f" (UID: "1f4dbe9b-0326-4112-b64f-003ba69ac14f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:46:50.620944 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:50.620897    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f4dbe9b-0326-4112-b64f-003ba69ac14f-maas-api-tls" pod "1f4dbe9b-0326-4112-b64f-003ba69ac14f" (UID: "1f4dbe9b-0326-4112-b64f-003ba69ac14f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:46:50.622432 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:50.622392    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f4dbe9b-0326-4112-b64f-003ba69ac14f-metrics-certs" pod "1f4dbe9b-0326-4112-b64f-003ba69ac14f" (UID: "1f4dbe9b-0326-4112-b64f-003ba69ac14f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:46:50.622957 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:50.622910    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f4dbe9b-0326-4112-b64f-003ba69ac14f-kube-api-access-kps68" pod "1f4dbe9b-0326-4112-b64f-003ba69ac14f" (UID: "1f4dbe9b-0326-4112-b64f-003ba69ac14f"). InnerVolumeSpecName "kube-api-access-kps68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:46:50.719654 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:50.717335    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4dbe9b-0326-4112-b64f-003ba69ac14f-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:46:50.719654 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:50.717374    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f4dbe9b-0326-4112-b64f-003ba69ac14f-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:46:50.719654 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:50.717388    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1f4dbe9b-0326-4112-b64f-003ba69ac14f-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:46:50.719654 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:50.717399    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kps68\" (UniqueName: \"kubernetes.io/projected/1f4dbe9b-0326-4112-b64f-003ba69ac14f-kube-api-access-kps68\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:46:51.238839 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:51.238800    4945 generic.go:356] "Generic (PLEG): container finished" podID="1f4dbe9b-0326-4112-b64f-003ba69ac14f" containerID="fe9eab2cc08a1fd541d8f8c7b46d9b39dec2923c9e055be9f6c64df73b909c62" exitCode=0
Oct 30 12:46:51.239260 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:51.238843    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"
Oct 30 12:46:51.239260 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:51.238879    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f" event={"ID":"1f4dbe9b-0326-4112-b64f-003ba69ac14f","Type":"ContainerDied","Data":"fe9eab2cc08a1fd541d8f8c7b46d9b39dec2923c9e055be9f6c64df73b909c62"}
Oct 30 12:46:51.239260 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:51.238937    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f" event={"ID":"1f4dbe9b-0326-4112-b64f-003ba69ac14f","Type":"ContainerDied","Data":"65a7dfde05cc19abe6fe2e94b31848517513305b1580280a1ba5432b67afe629"}
Oct 30 12:46:51.239260 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:51.238966    4945 scope.go:122] "RemoveContainer" containerID="fe9eab2cc08a1fd541d8f8c7b46d9b39dec2923c9e055be9f6c64df73b909c62"
Oct 30 12:46:51.249008 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:51.248992    4945 scope.go:122] "RemoveContainer" containerID="fe9eab2cc08a1fd541d8f8c7b46d9b39dec2923c9e055be9f6c64df73b909c62"
Oct 30 12:46:51.249268 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:46:51.249248    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe9eab2cc08a1fd541d8f8c7b46d9b39dec2923c9e055be9f6c64df73b909c62\": container with ID starting with fe9eab2cc08a1fd541d8f8c7b46d9b39dec2923c9e055be9f6c64df73b909c62 not found: ID does not exist" containerID="fe9eab2cc08a1fd541d8f8c7b46d9b39dec2923c9e055be9f6c64df73b909c62"
Oct 30 12:46:51.249374 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:51.249274    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe9eab2cc08a1fd541d8f8c7b46d9b39dec2923c9e055be9f6c64df73b909c62"} err="failed to get container status \"fe9eab2cc08a1fd541d8f8c7b46d9b39dec2923c9e055be9f6c64df73b909c62\": rpc error: code = NotFound desc = could not find container \"fe9eab2cc08a1fd541d8f8c7b46d9b39dec2923c9e055be9f6c64df73b909c62\": container with ID starting with fe9eab2cc08a1fd541d8f8c7b46d9b39dec2923c9e055be9f6c64df73b909c62 not found: ID does not exist"
Oct 30 12:46:51.286930 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:51.286901    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"]
Oct 30 12:46:51.290903 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:51.290881    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-48623dd7-5d68fddbc9-xhl2f"]
Oct 30 12:46:51.548407 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:51.548328    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f4dbe9b-0326-4112-b64f-003ba69ac14f" path="/var/lib/kubelet/pods/1f4dbe9b-0326-4112-b64f-003ba69ac14f/volumes"
Oct 30 12:46:52.896688 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:52.896655    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"]
Oct 30 12:46:52.897104 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:52.897090    4945 cpu_manager.go:414] "containerMap: removing container" podUID="1f4dbe9b-0326-4112-b64f-003ba69ac14f" containerName="maas-api"
Oct 30 12:46:52.897146 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:52.897104    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f4dbe9b-0326-4112-b64f-003ba69ac14f" containerName="maas-api"
Oct 30 12:46:52.897146 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:52.897134    4945 cpu_manager.go:414] "containerMap: removing container" podUID="0e6959df-26a0-4340-bc8d-4add6a0a798f" containerName="maas-api"
Oct 30 12:46:52.897146 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:52.897139    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e6959df-26a0-4340-bc8d-4add6a0a798f" containerName="maas-api"
Oct 30 12:46:52.897240 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:52.897206    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e6959df-26a0-4340-bc8d-4add6a0a798f" containerName="maas-api"
Oct 30 12:46:52.897240 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:52.897216    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f4dbe9b-0326-4112-b64f-003ba69ac14f" containerName="maas-api"
Oct 30 12:46:52.901575 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:52.901557    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:52.930264 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:52.930228    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-53dcb1-serving-cert\""
Oct 30 12:46:52.937655 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:52.937623    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"]
Oct 30 12:46:52.939895 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:52.939870    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpfpk\" (UniqueName: \"kubernetes.io/projected/15f559f9-e4fa-43f3-8308-a0c9a947080c-kube-api-access-mpfpk\") pod \"maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:52.940055 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:52.939924    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-maas-api-tls\") pod \"maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:52.940133 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:52.940051    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15f559f9-e4fa-43f3-8308-a0c9a947080c-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:52.940133 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:52.940079    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-metrics-certs\") pod \"maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:53.040773 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:53.040733    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpfpk\" (UniqueName: \"kubernetes.io/projected/15f559f9-e4fa-43f3-8308-a0c9a947080c-kube-api-access-mpfpk\") pod \"maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:53.040938 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:53.040803    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-maas-api-tls\") pod \"maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:53.040938 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:53.040872    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15f559f9-e4fa-43f3-8308-a0c9a947080c-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:53.041016 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:46:53.040958    4945 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-serving-cert: secret "maas-api-e2e-worker-w4-53dcb1-serving-cert" not found
Oct 30 12:46:53.041053 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:46:53.041026    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-maas-api-tls podName:15f559f9-e4fa-43f3-8308-a0c9a947080c nodeName:}" failed. No retries permitted until 2026-10-30 12:46:53.541007837 +0000 UTC m=+658.118956904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-maas-api-tls") pod "maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p" (UID: "15f559f9-e4fa-43f3-8308-a0c9a947080c") : secret "maas-api-e2e-worker-w4-53dcb1-serving-cert" not found
Oct 30 12:46:53.041053 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:53.040967    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-metrics-certs\") pod \"maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:53.041524 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:53.041498    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15f559f9-e4fa-43f3-8308-a0c9a947080c-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:53.043274 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:53.043236    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-metrics-certs\") pod \"maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:53.059974 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:53.059946    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpfpk\" (UniqueName: \"kubernetes.io/projected/15f559f9-e4fa-43f3-8308-a0c9a947080c-kube-api-access-mpfpk\") pod \"maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:53.546207 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:53.546166    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-maas-api-tls\") pod \"maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:53.548658 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:53.548594    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-maas-api-tls\") pod \"maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:53.816027 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:53.815949    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:54.113585 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:54.113499    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"]
Oct 30 12:46:54.118631 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:46:54.118585    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15f559f9_e4fa_43f3_8308_a0c9a947080c.slice/crio-2fe8d9c8d7f9bc47614dc63eb31425e8a3aeca0bb11e9f9fe6015eb45f3f7b45 WatchSource:0}: Error finding container 2fe8d9c8d7f9bc47614dc63eb31425e8a3aeca0bb11e9f9fe6015eb45f3f7b45: Status 404 returned error can't find the container with id 2fe8d9c8d7f9bc47614dc63eb31425e8a3aeca0bb11e9f9fe6015eb45f3f7b45
Oct 30 12:46:54.269493 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:54.269450    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p" event={"ID":"15f559f9-e4fa-43f3-8308-a0c9a947080c","Type":"ContainerStarted","Data":"2fe8d9c8d7f9bc47614dc63eb31425e8a3aeca0bb11e9f9fe6015eb45f3f7b45"}
Oct 30 12:46:55.279420 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:55.279384    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p" event={"ID":"15f559f9-e4fa-43f3-8308-a0c9a947080c","Type":"ContainerStarted","Data":"d5a97327a36cf488c6a84349c9fee4cdce5040ac0202e8c5d8f6b2bb253cdbbc"}
Oct 30 12:46:55.280010 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:55.279447    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:46:55.323259 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:46:55.323209    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p" podStartSLOduration=2.977987001 podStartE2EDuration="3.32319485s" totalImagesPullingTime="345.207849ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:46:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:46:55.321981233 +0000 UTC m=+659.899930323" watchObservedRunningTime="2026-10-30 12:46:55.32319485 +0000 UTC m=+659.901143938"
Oct 30 12:46:56.193235 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:46:56.193195    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:47:01.290822 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:01.290793    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 12:47:02.646274 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.646240    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"]
Oct 30 12:47:02.651584 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.651565    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:02.671907 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.671880    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-4cd3fb-serving-cert\""
Oct 30 12:47:02.695476 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.695441    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"]
Oct 30 12:47:02.741790 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.741756    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-maas-api-tls\") pod \"maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:02.741941 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.741794    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8qwx\" (UniqueName: \"kubernetes.io/projected/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-kube-api-access-s8qwx\") pod \"maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:02.741984 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.741943    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-metrics-certs\") pod \"maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:02.742019 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.741995    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:02.842802 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.842770    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-metrics-certs\") pod \"maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:02.842968 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.842827    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:02.842968 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.842882    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-maas-api-tls\") pod \"maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:02.842968 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.842908    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8qwx\" (UniqueName: \"kubernetes.io/projected/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-kube-api-access-s8qwx\") pod \"maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:02.843743 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.843712    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:02.845424 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.845397    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-maas-api-tls\") pod \"maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:02.845542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.845437    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-metrics-certs\") pod \"maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:02.867094 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.867069    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8qwx\" (UniqueName: \"kubernetes.io/projected/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-kube-api-access-s8qwx\") pod \"maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:02.964519 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:02.964468    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:03.275256 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:03.275183    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"]
Oct 30 12:47:03.278476 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:47:03.278445    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod120ca3bc_b398_43fa_b2d1_1dfa81002f9c.slice/crio-5240e230a88baecebce2552a83a81049735c291627d167c8c8a257b4ef097a57 WatchSource:0}: Error finding container 5240e230a88baecebce2552a83a81049735c291627d167c8c8a257b4ef097a57: Status 404 returned error can't find the container with id 5240e230a88baecebce2552a83a81049735c291627d167c8c8a257b4ef097a57
Oct 30 12:47:03.332281 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:03.332250    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx" event={"ID":"120ca3bc-b398-43fa-b2d1-1dfa81002f9c","Type":"ContainerStarted","Data":"5240e230a88baecebce2552a83a81049735c291627d167c8c8a257b4ef097a57"}
Oct 30 12:47:04.340343 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:04.340305    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx" event={"ID":"120ca3bc-b398-43fa-b2d1-1dfa81002f9c","Type":"ContainerStarted","Data":"f6007c3b8cd5cc4d0aaa187a3c62c62ece682dd02f99ff63d3094361775f248c"}
Oct 30 12:47:04.340763 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:04.340429    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:04.381647 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:04.381573    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx" podStartSLOduration=2.044773867 podStartE2EDuration="2.38155945s" totalImagesPullingTime="336.785583ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:47:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:47:04.379874431 +0000 UTC m=+668.957823521" watchObservedRunningTime="2026-10-30 12:47:04.38155945 +0000 UTC m=+668.959508540"
Oct 30 12:47:10.353780 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:10.353545    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 12:47:26.195585 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:47:26.195538    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:47:30.181824 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.181791    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"]
Oct 30 12:47:30.185866 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.185843    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.230301 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.230268    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-home\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.230462 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.230324    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4cd3fb\"/\"openshift-service-ca.crt\""
Oct 30 12:47:30.230462 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.230325    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41d6c6e-7427-499e-b4b5-b69590e5a458-tls-certs\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.230462 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.230395    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftlwg\" (UniqueName: \"kubernetes.io/projected/b41d6c6e-7427-499e-b4b5-b69590e5a458-kube-api-access-ftlwg\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.230462 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.230419    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-model-cache\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.230646 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.230492    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-dshm\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.230646 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.230523    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-tmp-dir\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.242118 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.242070    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-4cd3fb\"/\"kube-root-ca.crt\""
Oct 30 12:47:30.242272 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.242153    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4cd3fb\"/\"default-dockercfg-slm6x\""
Oct 30 12:47:30.242364 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.242345    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4cd3fb\"/\"simulator-w0-4cd3fb-kserve-self-signed-certs\""
Oct 30 12:47:30.245018 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.244994    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"]
Oct 30 12:47:30.331138 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.331097    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-dshm\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.331342 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.331165    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-tmp-dir\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.331342 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.331245    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-home\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.331342 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.331277    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41d6c6e-7427-499e-b4b5-b69590e5a458-tls-certs\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.331342 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.331310    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftlwg\" (UniqueName: \"kubernetes.io/projected/b41d6c6e-7427-499e-b4b5-b69590e5a458-kube-api-access-ftlwg\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.331342 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.331341    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-model-cache\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.332140 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.332116    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-model-cache\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.332322 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.332176    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-tmp-dir\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.332322 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.332241    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-home\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.334488 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.334462    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41d6c6e-7427-499e-b4b5-b69590e5a458-tls-certs\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.334765 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.334738    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-dshm\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.351770 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.351737    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftlwg\" (UniqueName: \"kubernetes.io/projected/b41d6c6e-7427-499e-b4b5-b69590e5a458-kube-api-access-ftlwg\") pod \"simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") " pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.499333 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.499298    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:30.805175 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:30.805146    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"]
Oct 30 12:47:30.807510 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:47:30.807485    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb41d6c6e_7427_499e_b4b5_b69590e5a458.slice/crio-257f5ebe2e9e9c4778f2e02c0ba881902b12c0c2031a318207f41bd3ccc15e8a WatchSource:0}: Error finding container 257f5ebe2e9e9c4778f2e02c0ba881902b12c0c2031a318207f41bd3ccc15e8a: Status 404 returned error can't find the container with id 257f5ebe2e9e9c4778f2e02c0ba881902b12c0c2031a318207f41bd3ccc15e8a
Oct 30 12:47:31.507749 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:31.507710    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" event={"ID":"b41d6c6e-7427-499e-b4b5-b69590e5a458","Type":"ContainerStarted","Data":"257f5ebe2e9e9c4778f2e02c0ba881902b12c0c2031a318207f41bd3ccc15e8a"}
Oct 30 12:47:32.516615 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:32.516553    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" event={"ID":"b41d6c6e-7427-499e-b4b5-b69590e5a458","Type":"ContainerStarted","Data":"b04432ed907e257cc1efd069344226762166c83900ad06797c1be3eb7f015d9b"}
Oct 30 12:47:32.589592 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:32.589532    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podStartSLOduration=1.532292105 podStartE2EDuration="2.589513788s" totalImagesPullingTime="1.057221683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:47:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:47:32.588276764 +0000 UTC m=+697.166225853" watchObservedRunningTime="2026-10-30 12:47:32.589513788 +0000 UTC m=+697.167462882"
Oct 30 12:47:33.500461 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:33.500421    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:40.500319 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:40.500279    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:40.513060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:40.513037    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:40.575772 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:40.575743    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:47:42.638340 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.638307    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"]
Oct 30 12:47:42.645283 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.645265    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.676413 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.676377    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-7a590a\"/\"openshift-service-ca.crt\""
Oct 30 12:47:42.676413 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.676384    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-7a590a\"/\"kube-root-ca.crt\""
Oct 30 12:47:42.676589 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.676487    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-7a590a\"/\"simulator-w1-7a590a-kserve-self-signed-certs\""
Oct 30 12:47:42.677580 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.677559    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-7a590a\"/\"default-dockercfg-lzwt6\""
Oct 30 12:47:42.679019 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.678995    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"]
Oct 30 12:47:42.760501 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.760465    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plmpv\" (UniqueName: \"kubernetes.io/projected/0a699888-47fc-44b7-88fb-26b4a0efecba-kube-api-access-plmpv\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.760677 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.760511    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-dshm\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.760677 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.760538    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-model-cache\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.760781 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.760694    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a699888-47fc-44b7-88fb-26b4a0efecba-tls-certs\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.760781 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.760730    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-tmp-dir\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.760849 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.760816    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-home\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.861334 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.861296    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a699888-47fc-44b7-88fb-26b4a0efecba-tls-certs\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.861334 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.861346    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-tmp-dir\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.861635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.861378    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-home\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.861635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.861418    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plmpv\" (UniqueName: \"kubernetes.io/projected/0a699888-47fc-44b7-88fb-26b4a0efecba-kube-api-access-plmpv\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.861635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.861444    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-dshm\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.861635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.861461    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-model-cache\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.861921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.861895    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-tmp-dir\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.862183 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.862155    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-home\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.862460 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.862436    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-model-cache\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.863793 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.863771    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-dshm\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.864077 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.864055    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a699888-47fc-44b7-88fb-26b4a0efecba-tls-certs\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.885105 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.885078    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plmpv\" (UniqueName: \"kubernetes.io/projected/0a699888-47fc-44b7-88fb-26b4a0efecba-kube-api-access-plmpv\") pod \"simulator-w1-7a590a-kserve-676464b495-kq85h\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") " pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:42.957933 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:42.957900    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:43.292216 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:43.292187    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"]
Oct 30 12:47:43.294449 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:47:43.294421    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a699888_47fc_44b7_88fb_26b4a0efecba.slice/crio-548450e24277a501171ad82b866e4e0b28309fbbd4c5a2b2965e490ca93580fe WatchSource:0}: Error finding container 548450e24277a501171ad82b866e4e0b28309fbbd4c5a2b2965e490ca93580fe: Status 404 returned error can't find the container with id 548450e24277a501171ad82b866e4e0b28309fbbd4c5a2b2965e490ca93580fe
Oct 30 12:47:43.584736 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:43.584650    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" event={"ID":"0a699888-47fc-44b7-88fb-26b4a0efecba","Type":"ContainerStarted","Data":"0e7735c46ecb3cc5627a5d794a086a8b6a3b712d99b0b6b05553030bea8367d4"}
Oct 30 12:47:43.584736 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:43.584700    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" event={"ID":"0a699888-47fc-44b7-88fb-26b4a0efecba","Type":"ContainerStarted","Data":"548450e24277a501171ad82b866e4e0b28309fbbd4c5a2b2965e490ca93580fe"}
Oct 30 12:47:43.631149 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:43.631095    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podStartSLOduration=1.631077591 podStartE2EDuration="1.631077591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:47:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:47:43.6297393 +0000 UTC m=+708.207688389" watchObservedRunningTime="2026-10-30 12:47:43.631077591 +0000 UTC m=+708.209026681"
Oct 30 12:47:43.958321 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:43.958294    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:52.958862 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:52.958828    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:52.973948 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:52.973921    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:53.652208 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:53.652177    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:47:56.196364 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:47:56.196319    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:47:58.478672 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.478632    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"]
Oct 30 12:47:58.484126 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.484101    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.521287 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.521255    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"]
Oct 30 12:47:58.529931 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.529905    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-4cd3fb\"/\"premium-w0-4cd3fb-kserve-self-signed-certs\""
Oct 30 12:47:58.626505 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.626464    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-tmp-dir\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.626715 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.626589    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-model-cache\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.626715 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.626640    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-home\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.626715 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.626675    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-dshm\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.626893 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.626755    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-tls-certs\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.626893 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.626795    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbbq7\" (UniqueName: \"kubernetes.io/projected/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-kube-api-access-xbbq7\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.728514 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.728480    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-tmp-dir\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.729367 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.729299    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-model-cache\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.729367 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.729341    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-home\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.729561 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.729373    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-dshm\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.729561 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.729411    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-tls-certs\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.729561 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.729443    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbbq7\" (UniqueName: \"kubernetes.io/projected/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-kube-api-access-xbbq7\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.729812 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.729165    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-tmp-dir\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.730078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.730056    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-model-cache\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.730310 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.730291    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-home\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.734384 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.734142    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-dshm\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.735275 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.735249    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-tls-certs\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.753216 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.753186    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbbq7\" (UniqueName: \"kubernetes.io/projected/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-kube-api-access-xbbq7\") pod \"premium-w0-4cd3fb-kserve-6696787bf8-dxpl6\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") " pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:58.812434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:58.812399    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:47:59.145795 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:59.145771    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"]
Oct 30 12:47:59.148282 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:47:59.148252    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod819b89b0_600a_4bb8_8a5c_adaa0d941c7e.slice/crio-478639405ba5b2ad38aede293777d1ab297a00ab5ec26e7bc9569ed661c1fe26 WatchSource:0}: Error finding container 478639405ba5b2ad38aede293777d1ab297a00ab5ec26e7bc9569ed661c1fe26: Status 404 returned error can't find the container with id 478639405ba5b2ad38aede293777d1ab297a00ab5ec26e7bc9569ed661c1fe26
Oct 30 12:47:59.676884 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:59.676847    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" event={"ID":"819b89b0-600a-4bb8-8a5c-adaa0d941c7e","Type":"ContainerStarted","Data":"c87ca46100e3384d9043ea7b67da5057f9aa2a8655556b09794129ac8a65e31b"}
Oct 30 12:47:59.677273 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:59.676895    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" event={"ID":"819b89b0-600a-4bb8-8a5c-adaa0d941c7e","Type":"ContainerStarted","Data":"478639405ba5b2ad38aede293777d1ab297a00ab5ec26e7bc9569ed661c1fe26"}
Oct 30 12:47:59.718124 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:59.718067    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podStartSLOduration=1.718051772 podStartE2EDuration="1.718051772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:47:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:47:59.716454144 +0000 UTC m=+724.294403214" watchObservedRunningTime="2026-10-30 12:47:59.718051772 +0000 UTC m=+724.296000861"
Oct 30 12:47:59.812873 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:47:59.812839    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:48:07.235978 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.235945    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"]
Oct 30 12:48:07.240046 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.240027    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.268557 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.268528    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"]
Oct 30 12:48:07.345118 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.345087    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-4cd3fb-openshift-default-dockercfg-sd289\""
Oct 30 12:48:07.420167 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.420131    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-workload-certs\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.420340 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.420176    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-podinfo\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.420340 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.420298    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-data\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.420431 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.420348    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-credential-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.420472 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.420426    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-envoy\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.420507 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.420478    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0ba9b13-4372-4d8f-9916-30246cc34e84-istiod-ca-cert\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.420547 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.420524    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-workload-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.420594 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.420580    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-token\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.420651 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.420619    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl7qz\" (UniqueName: \"kubernetes.io/projected/d0ba9b13-4372-4d8f-9916-30246cc34e84-kube-api-access-kl7qz\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.521755 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.521676    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-podinfo\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.521755 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.521727    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-data\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.521755 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.521750    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-credential-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.522030 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.521784    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-envoy\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.522030 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.521822    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0ba9b13-4372-4d8f-9916-30246cc34e84-istiod-ca-cert\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.522030 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.521970    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-workload-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.522126 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.522050    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-token\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.522126 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.522081    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl7qz\" (UniqueName: \"kubernetes.io/projected/d0ba9b13-4372-4d8f-9916-30246cc34e84-kube-api-access-kl7qz\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.522189 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.522151    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-workload-certs\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.522540 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.522512    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-workload-certs\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.522688 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.522567    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0ba9b13-4372-4d8f-9916-30246cc34e84-istiod-ca-cert\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.522688 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.522629    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-workload-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.522688 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.522680    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-data\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.522885 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.522859    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-credential-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.524665 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.524588    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-envoy\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.524665 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.524657    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-podinfo\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.543255 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.543226    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-token\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.543941 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.543918    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl7qz\" (UniqueName: \"kubernetes.io/projected/d0ba9b13-4372-4d8f-9916-30246cc34e84-kube-api-access-kl7qz\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.554395 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.554371    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:07.869176 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.869150    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"]
Oct 30 12:48:07.871989 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:48:07.871956    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ba9b13_4372_4d8f_9916_30246cc34e84.slice/crio-ca6a4928a942a152525ec4c085067dc80369841ece2c47a5a36752cdcf72ab33 WatchSource:0}: Error finding container ca6a4928a942a152525ec4c085067dc80369841ece2c47a5a36752cdcf72ab33: Status 404 returned error can't find the container with id ca6a4928a942a152525ec4c085067dc80369841ece2c47a5a36752cdcf72ab33
Oct 30 12:48:07.874565 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.874527    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:07.874682 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.874664    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:07.874724 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:07.874711    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:08.736546 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:08.736507    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" event={"ID":"d0ba9b13-4372-4d8f-9916-30246cc34e84","Type":"ContainerStarted","Data":"ad3c4e5a8636a349179dd069c1ab44c58110fba3a489619edb9dfde8ee613268"}
Oct 30 12:48:08.736964 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:08.736561    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" event={"ID":"d0ba9b13-4372-4d8f-9916-30246cc34e84","Type":"ContainerStarted","Data":"ca6a4928a942a152525ec4c085067dc80369841ece2c47a5a36752cdcf72ab33"}
Oct 30 12:48:08.805018 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:08.804956    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" podStartSLOduration=1.8049397520000001 podStartE2EDuration="1.804939752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:48:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:48:08.792257427 +0000 UTC m=+733.370206517" watchObservedRunningTime="2026-10-30 12:48:08.804939752 +0000 UTC m=+733.382888901"
Oct 30 12:48:08.812786 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:08.812720    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:48:08.832490 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:08.832458    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:48:09.566333 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:09.566293    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:09.568126 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:09.567970    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 30 12:48:09.568126 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:09.568035    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 30 12:48:09.754365 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:09.754336    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:48:10.555201 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:10.555165    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 30 12:48:10.555369 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:10.555217    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 30 12:48:11.554920 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:11.554884    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 30 12:48:11.555399 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:11.554943    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 30 12:48:12.554881 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:12.554845    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 30 12:48:12.555049 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:12.554916    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 30 12:48:13.554886 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:13.554853    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 30 12:48:13.555051 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:13.554905    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 30 12:48:15.599966 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:15.597701    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:48:15.599966 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:15.597767    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:48:15.699107 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:15.699064    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:15.699586 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:15.699562    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:15.700426 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:15.700406    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 12:48:20.724032 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.723994    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"]
Oct 30 12:48:20.729840 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.729813    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.756086 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.756057    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-7a590a\"/\"premium-w1-7a590a-kserve-self-signed-certs\""
Oct 30 12:48:20.758884 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.758859    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"]
Oct 30 12:48:20.804525 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.804485    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-model-cache\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.804736 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.804533    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-dshm\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.804736 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.804640    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-tls-certs\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.804736 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.804697    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-tmp-dir\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.804917 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.804752    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-home\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.804917 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.804774    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flp5b\" (UniqueName: \"kubernetes.io/projected/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-kube-api-access-flp5b\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.906041 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.905999    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-tmp-dir\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.906239 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.906107    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-home\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.906239 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.906137    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flp5b\" (UniqueName: \"kubernetes.io/projected/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-kube-api-access-flp5b\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.906239 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.906191    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-model-cache\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.906239 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.906229    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-dshm\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.906433 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.906254    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-tls-certs\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.906503 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.906473    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-tmp-dir\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.906862 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.906837    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-model-cache\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.907151 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.907129    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-home\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.913204 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.913180    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-tls-certs\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.914872 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.914840    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-dshm\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:20.932568 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:20.932505    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flp5b\" (UniqueName: \"kubernetes.io/projected/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-kube-api-access-flp5b\") pod \"premium-w1-7a590a-kserve-5cb8dddf94-8dhkh\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") " pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:21.045303 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:21.045217    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:21.345037 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:21.344961    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"]
Oct 30 12:48:21.348330 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:48:21.348302    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a1454c3_5b47_4a17_8dd9_dc9ba7735caa.slice/crio-b58077487caf479458785f6789370b566b7357a91c21f8eb2997ee10d9864349 WatchSource:0}: Error finding container b58077487caf479458785f6789370b566b7357a91c21f8eb2997ee10d9864349: Status 404 returned error can't find the container with id b58077487caf479458785f6789370b566b7357a91c21f8eb2997ee10d9864349
Oct 30 12:48:21.841690 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:21.841654    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" event={"ID":"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa","Type":"ContainerStarted","Data":"5e49c961322088e3f44fabad7357e18fe9c29383abccd865804c0ed0c0402713"}
Oct 30 12:48:21.842149 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:21.841705    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" event={"ID":"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa","Type":"ContainerStarted","Data":"b58077487caf479458785f6789370b566b7357a91c21f8eb2997ee10d9864349"}
Oct 30 12:48:22.046036 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:22.045834    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:25.845216 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:25.845155    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podStartSLOduration=5.845136612 podStartE2EDuration="5.845136612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:48:21.891851725 +0000 UTC m=+746.469800815" watchObservedRunningTime="2026-10-30 12:48:25.845136612 +0000 UTC m=+750.423085702"
Oct 30 12:48:25.847706 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:25.847515    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"]
Oct 30 12:48:25.852735 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:25.852700    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:25.881943 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:25.881919    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-c81f42-openshift-default-dockercfg-7fkzs\""
Oct 30 12:48:25.890622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:25.889893    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"]
Oct 30 12:48:25.972143 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:25.972107    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-podinfo\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:25.972318 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:25.972157    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-data\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:25.972318 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:25.972260    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djf9q\" (UniqueName: \"kubernetes.io/projected/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-kube-api-access-djf9q\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:25.972318 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:25.972312    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-credential-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:25.972506 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:25.972479    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-workload-certs\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:25.972593 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:25.972579    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-envoy\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:25.972593 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:25.972627    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istiod-ca-cert\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:25.972757 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:25.972697    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-token\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:25.972798 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:25.972750    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-workload-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.073533 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.073499    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-envoy\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.073533 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.073535    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istiod-ca-cert\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.073769 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.073570    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-token\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.073769 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.073627    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-workload-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.074042 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.074015    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-podinfo\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.074257 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.074237    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-data\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.074410 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.074391    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djf9q\" (UniqueName: \"kubernetes.io/projected/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-kube-api-access-djf9q\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.074542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.074516    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-data\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.074640 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.074270    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-workload-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.074640 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.074401    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istiod-ca-cert\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.074787 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.074526    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-credential-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.074891 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.074867    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-credential-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.074995 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.074968    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-workload-certs\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.075345 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.075322    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-workload-certs\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.076308 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.076284    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-envoy\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.076559 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.076540    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-podinfo\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.096178 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.096115    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-token\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.097703 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.097683    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djf9q\" (UniqueName: \"kubernetes.io/projected/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-kube-api-access-djf9q\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.192957 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:48:26.192916    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:48:26.197282 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.197253    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:26.434159 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.434079    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"]
Oct 30 12:48:26.439347 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.439328    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.454418 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.454389    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-105228-openshift-default-dockercfg-kvxgn\""
Oct 30 12:48:26.465181 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.465157    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"]
Oct 30 12:48:26.529520 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:48:26.529486    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9b9135f_faa3_4b2c_ba00_d381772f4ea7.slice/crio-0448043a9358f468297aa947723013170e789c0d961d42a21a67a276cad33761 WatchSource:0}: Error finding container 0448043a9358f468297aa947723013170e789c0d961d42a21a67a276cad33761: Status 404 returned error can't find the container with id 0448043a9358f468297aa947723013170e789c0d961d42a21a67a276cad33761
Oct 30 12:48:26.531757 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.531732    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"]
Oct 30 12:48:26.532089 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.532063    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:26.532141 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.532119    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:26.532175 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.532146    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:26.580531 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.580504    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfqmj\" (UniqueName: \"kubernetes.io/projected/a069f35f-76b4-4583-857a-304896b3f33d-kube-api-access-rfqmj\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.580675 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.580553    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-credential-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.580747 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.580730    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-workload-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.580806 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.580773    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a069f35f-76b4-4583-857a-304896b3f33d-istio-token\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.580860 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.580843    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-istio-data\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.580912 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.580869    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-workload-certs\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.581016 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.580994    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-istio-envoy\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.581069 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.581048    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a069f35f-76b4-4583-857a-304896b3f33d-istio-podinfo\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.581138 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.581089    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a069f35f-76b4-4583-857a-304896b3f33d-istiod-ca-cert\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.682949 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.682296    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-istio-envoy\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.682949 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.682348    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a069f35f-76b4-4583-857a-304896b3f33d-istio-podinfo\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.682949 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.682434    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a069f35f-76b4-4583-857a-304896b3f33d-istiod-ca-cert\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.682949 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.682491    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfqmj\" (UniqueName: \"kubernetes.io/projected/a069f35f-76b4-4583-857a-304896b3f33d-kube-api-access-rfqmj\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.682949 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.682554    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-credential-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.682949 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.682645    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-workload-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.682949 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.682682    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a069f35f-76b4-4583-857a-304896b3f33d-istio-token\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.682949 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.682739    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-istio-data\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.682949 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.682769    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-workload-certs\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.683948 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.683004    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-credential-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.683948 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.683129    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-workload-certs\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.683948 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.683384    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-workload-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.683948 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.683652    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a069f35f-76b4-4583-857a-304896b3f33d-istiod-ca-cert\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.683948 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.683928    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-istio-data\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.685524 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.685438    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a069f35f-76b4-4583-857a-304896b3f33d-istio-podinfo\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.685926 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.685906    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-istio-envoy\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.703335 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.703310    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a069f35f-76b4-4583-857a-304896b3f33d-istio-token\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.703441 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.703405    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfqmj\" (UniqueName: \"kubernetes.io/projected/a069f35f-76b4-4583-857a-304896b3f33d-kube-api-access-rfqmj\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.753506 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.753474    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:26.877663 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.877623    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" event={"ID":"a9b9135f-faa3-4b2c-ba00-d381772f4ea7","Type":"ContainerStarted","Data":"0149777bc760a9584c3f6ec950064b7e65f3cf1829d6af7b991f5f5d2b2c780f"}
Oct 30 12:48:26.878116 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.877692    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" event={"ID":"a9b9135f-faa3-4b2c-ba00-d381772f4ea7","Type":"ContainerStarted","Data":"0448043a9358f468297aa947723013170e789c0d961d42a21a67a276cad33761"}
Oct 30 12:48:26.925529 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:26.925472    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" podStartSLOduration=1.925455646 podStartE2EDuration="1.925455646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:48:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:48:26.919934066 +0000 UTC m=+751.497883156" watchObservedRunningTime="2026-10-30 12:48:26.925455646 +0000 UTC m=+751.503404736"
Oct 30 12:48:27.093532 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:27.093499    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"]
Oct 30 12:48:27.095551 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:27.095512    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:27.096219 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:27.095618    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:27.096219 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:27.095663    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:27.197508 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:27.197476    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:27.198733 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:27.198704    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 30 12:48:27.198858 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:27.198760    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" podUID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 30 12:48:27.917859 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:27.917805    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" event={"ID":"a069f35f-76b4-4583-857a-304896b3f33d","Type":"ContainerStarted","Data":"5dc0da14ca6524d69fb02ff8d95d6e069cf386f9fc824347b453d2fb5bc9d370"}
Oct 30 12:48:27.918378 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:27.917920    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" event={"ID":"a069f35f-76b4-4583-857a-304896b3f33d","Type":"ContainerStarted","Data":"0a3b4e5b23b49f3bce43d391d6aae7597c98cafa9dad6a31c32a1b7805dd3611"}
Oct 30 12:48:27.982792 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:27.982728    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" podStartSLOduration=1.9827074329999999 podStartE2EDuration="1.982707433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:48:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:48:27.968116246 +0000 UTC m=+752.546065337" watchObservedRunningTime="2026-10-30 12:48:27.982707433 +0000 UTC m=+752.560656526"
Oct 30 12:48:28.198057 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:28.198021    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 30 12:48:28.198378 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:28.198348    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" podUID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 30 12:48:28.753926 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:28.753890    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:28.755093 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:28.755068    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 30 12:48:28.755202 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:28.755127    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" podUID="a069f35f-76b4-4583-857a-304896b3f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 30 12:48:29.200004 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:29.199966    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 30 12:48:29.200975 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:29.200572    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" podUID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 30 12:48:29.755530 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:29.755496    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 30 12:48:29.755749 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:29.755672    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" podUID="a069f35f-76b4-4583-857a-304896b3f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 30 12:48:30.200622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:30.197997    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 30 12:48:30.200622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:30.198059    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" podUID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 30 12:48:30.754503 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:30.754469    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 30 12:48:30.754709 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:30.754578    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" podUID="a069f35f-76b4-4583-857a-304896b3f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 30 12:48:31.052261 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.049801    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:31.069706 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.069676    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:31.115340 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.115306    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"]
Oct 30 12:48:31.150898 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.150868    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"]
Oct 30 12:48:31.151250 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.151190    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.189949 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.189919    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-53dcb1-openshift-default-dockercfg-nlgks\""
Oct 30 12:48:31.233302 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.233251    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 30 12:48:31.235528 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.234729    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" podUID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 30 12:48:31.333308 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.333207    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-credential-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.333990 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.333963    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-podinfo\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.348758 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.348724    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-token\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.349037 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.349014    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvm6v\" (UniqueName: \"kubernetes.io/projected/473f256f-7ee3-4e49-b407-346e0242fcbb-kube-api-access-dvm6v\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.349244 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.349226    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-data\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.349430 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.349410    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/473f256f-7ee3-4e49-b407-346e0242fcbb-istiod-ca-cert\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.349624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.349588    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-envoy\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.349820 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.349802    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-workload-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.350001 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.349984    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-workload-certs\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.451296 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.451259    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-credential-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.451296 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.451308    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-podinfo\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.451564 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.451358    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-token\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.451564 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.451404    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvm6v\" (UniqueName: \"kubernetes.io/projected/473f256f-7ee3-4e49-b407-346e0242fcbb-kube-api-access-dvm6v\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.451564 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.451441    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-data\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.451564 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.451471    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/473f256f-7ee3-4e49-b407-346e0242fcbb-istiod-ca-cert\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.451564 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.451503    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-envoy\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.451564 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.451547    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-workload-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.451877 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.451579    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-workload-certs\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.452167 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.452142    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-workload-certs\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.452556 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.452528    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-credential-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.452680 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.452647    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-data\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.452964 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.452927    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-workload-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.453405 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.453378    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/473f256f-7ee3-4e49-b407-346e0242fcbb-istiod-ca-cert\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.455149 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.455128    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-podinfo\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.455473 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.455453    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-envoy\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.482627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.481138    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-token\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.485352 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.485327    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvm6v\" (UniqueName: \"kubernetes.io/projected/473f256f-7ee3-4e49-b407-346e0242fcbb-kube-api-access-dvm6v\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:31.760978 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.760942    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 30 12:48:31.761259 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.761229    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" podUID="a069f35f-76b4-4583-857a-304896b3f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 30 12:48:31.768921 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:31.768883    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:32.046162 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:32.045837    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:48:32.210716 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:32.200018    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 30 12:48:32.210716 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:32.200081    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" podUID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 30 12:48:32.527892 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:32.523822    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"]
Oct 30 12:48:32.531050 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:32.531012    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:32.535619 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:32.531550    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:32.535821 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:32.535794    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:32.760629 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:32.754961    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 30 12:48:32.760629 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:32.755037    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" podUID="a069f35f-76b4-4583-857a-304896b3f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 30 12:48:32.976624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:32.972637    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" event={"ID":"473f256f-7ee3-4e49-b407-346e0242fcbb","Type":"ContainerStarted","Data":"8fbbe50b7d48c9089db28460562410ad41175fd9c24f0c5e69166d2dbd686a73"}
Oct 30 12:48:32.976624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:32.972738    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" event={"ID":"473f256f-7ee3-4e49-b407-346e0242fcbb","Type":"ContainerStarted","Data":"8e535f5c0556d744b166261960389f479f3558bb38ca5453e30a4bf6df84fb63"}
Oct 30 12:48:33.036683 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:33.036247    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" podStartSLOduration=2.03622867 podStartE2EDuration="2.03622867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:48:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:48:33.03474701 +0000 UTC m=+757.612696104" watchObservedRunningTime="2026-10-30 12:48:33.03622867 +0000 UTC m=+757.614177760"
Oct 30 12:48:33.206046 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:33.200700    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 30 12:48:33.206046 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:33.200773    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" podUID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 30 12:48:33.753984 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:33.753943    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 30 12:48:33.754507 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:33.754024    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" podUID="a069f35f-76b4-4583-857a-304896b3f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 30 12:48:33.772489 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:33.772455    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:33.772861 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:33.772837    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 30 12:48:33.773552 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:33.773519    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" podUID="473f256f-7ee3-4e49-b407-346e0242fcbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 30 12:48:34.770486 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:34.770447    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 30 12:48:34.771060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:34.770496    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" podUID="473f256f-7ee3-4e49-b407-346e0242fcbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 30 12:48:35.214892 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:35.214856    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:48:35.215204 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:35.215177    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" podUID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:48:35.259823 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:35.259788    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:35.260778 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:35.260724    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:35.261671 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:35.261624    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 12:48:35.770091 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:35.770060    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 30 12:48:35.770279 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:35.770121    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" podUID="473f256f-7ee3-4e49-b407-346e0242fcbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 30 12:48:35.802624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:35.800697    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:48:35.802624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:35.800757    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" podUID="a069f35f-76b4-4583-857a-304896b3f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:48:35.908627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:35.905880    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:35.908627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:35.907291    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:35.927953 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:35.927915    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 12:48:36.770074 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:36.769984    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 30 12:48:36.770074 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:36.770043    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" podUID="473f256f-7ee3-4e49-b407-346e0242fcbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 30 12:48:37.769833 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:37.769799    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 30 12:48:37.770256 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:37.769853    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" podUID="473f256f-7ee3-4e49-b407-346e0242fcbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 30 12:48:38.769951 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:38.769917    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 30 12:48:38.770387 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:38.769984    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" podUID="473f256f-7ee3-4e49-b407-346e0242fcbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 30 12:48:39.981247 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:39.981198    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:39.982044 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:39.982020    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:39.983088 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:39.983053    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 12:48:41.730763 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.730723    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"]
Oct 30 12:48:41.738938 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.738914    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.749823 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.749547    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-credential-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.749823 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.749590    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-workload-certs\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.749823 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.749666    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-podinfo\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.749823 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.749702    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istiod-ca-cert\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.749823 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.749746    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-envoy\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.750370 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.749854    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-data\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.750370 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.750048    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-workload-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.750370 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.750083    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd5qq\" (UniqueName: \"kubernetes.io/projected/81b43fdc-0af8-4443-a1ba-953d6ad604e4-kube-api-access-qd5qq\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.750370 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.750116    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-token\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.761192 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.761166    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"]
Oct 30 12:48:41.859879 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.859831    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-podinfo\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.860161 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.860137    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istiod-ca-cert\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.860298 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.860283    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-envoy\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.860436 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.860422    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-data\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.860613 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.860583    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-workload-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.860736 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.860719    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd5qq\" (UniqueName: \"kubernetes.io/projected/81b43fdc-0af8-4443-a1ba-953d6ad604e4-kube-api-access-qd5qq\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.860857 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.860844    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-token\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.861001 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.860989    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-credential-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.861088 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.861076    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-workload-certs\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.861574 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.861543    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-workload-certs\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.865982 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.865941    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-workload-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.866854 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.866808    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istiod-ca-cert\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.869348 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.869317    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-podinfo\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.875723 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.870266    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-credential-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.875723 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.870512    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-data\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.894869 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.894837    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-envoy\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.919616 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.895618    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd5qq\" (UniqueName: \"kubernetes.io/projected/81b43fdc-0af8-4443-a1ba-953d6ad604e4-kube-api-access-qd5qq\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:41.919616 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:41.911318    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-token\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:42.056182 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:42.056092    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:42.413236 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:42.413192    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"]
Oct 30 12:48:42.422538 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:42.422319    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:42.422538 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:42.422409    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:42.422538 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:42.422451    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:43.111404 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:43.111362    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" event={"ID":"81b43fdc-0af8-4443-a1ba-953d6ad604e4","Type":"ContainerStarted","Data":"807a53e7d415be7f6a80c6cd48990cc8a63a466bafa8b307d6e6a8a0ce1f7b49"}
Oct 30 12:48:43.112047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:43.112020    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" event={"ID":"81b43fdc-0af8-4443-a1ba-953d6ad604e4","Type":"ContainerStarted","Data":"a69f97de55635a58542247a81ae842d50d200fd91a9b4daa3be8fca7849e43ac"}
Oct 30 12:48:43.169863 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:43.169800    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" podStartSLOduration=2.169778183 podStartE2EDuration="2.169778183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:48:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:48:43.167407868 +0000 UTC m=+767.745356959" watchObservedRunningTime="2026-10-30 12:48:43.169778183 +0000 UTC m=+767.747727274"
Oct 30 12:48:44.056957 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:44.056928    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:44.058390 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:44.058358    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 30 12:48:44.058512 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:44.058406    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 30 12:48:45.056527 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:45.056483    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 30 12:48:45.056947 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:45.056544    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 30 12:48:46.056834 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:46.056802    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 30 12:48:46.057228 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:46.056852    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 30 12:48:47.057335 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:47.057296    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 30 12:48:47.057743 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:47.057375    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 30 12:48:48.056961 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:48.056924    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 30 12:48:48.057136 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:48.057001    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 30 12:48:49.056801 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:49.056767    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 30 12:48:49.057291 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:49.056826    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 30 12:48:50.061167 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:50.061134    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:50.061547 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:50.061510    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:50.062251 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:50.062228    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 12:48:52.382891 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.382848    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"]
Oct 30 12:48:52.389835 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.389812    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.418144 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.418115    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"]
Oct 30 12:48:52.516460 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.516423    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-workload-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.516656 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.516471    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-credential-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.516656 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.516565    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fb24330-4337-4c41-80d0-e780d94119a4-istiod-ca-cert\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.516656 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.516636    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fb24330-4337-4c41-80d0-e780d94119a4-istio-token\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.516810 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.516757    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-workload-certs\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.516810 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.516782    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f64wc\" (UniqueName: \"kubernetes.io/projected/4fb24330-4337-4c41-80d0-e780d94119a4-kube-api-access-f64wc\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.516899 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.516868    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fb24330-4337-4c41-80d0-e780d94119a4-istio-podinfo\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.516949 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.516912    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-istio-envoy\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.516949 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.516930    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-istio-data\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.617944 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.617913    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-istio-envoy\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.617944 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.617946    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-istio-data\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.618169 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.617972    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-workload-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.618169 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.617998    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-credential-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.618169 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.618034    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fb24330-4337-4c41-80d0-e780d94119a4-istiod-ca-cert\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.618169 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.618091    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fb24330-4337-4c41-80d0-e780d94119a4-istio-token\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.618375 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.618178    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-workload-certs\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.618375 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.618204    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f64wc\" (UniqueName: \"kubernetes.io/projected/4fb24330-4337-4c41-80d0-e780d94119a4-kube-api-access-f64wc\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.618375 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.618280    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fb24330-4337-4c41-80d0-e780d94119a4-istio-podinfo\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.618375 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.618359    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-istio-data\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.618587 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.618388    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-workload-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.618587 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.618493    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-credential-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.618821 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.618778    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-workload-certs\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.618998 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.618975    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fb24330-4337-4c41-80d0-e780d94119a4-istiod-ca-cert\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.620414 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.620393    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-istio-envoy\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.620961 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.620943    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fb24330-4337-4c41-80d0-e780d94119a4-istio-podinfo\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.638513 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.638447    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fb24330-4337-4c41-80d0-e780d94119a4-istio-token\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.638662 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.638643    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f64wc\" (UniqueName: \"kubernetes.io/projected/4fb24330-4337-4c41-80d0-e780d94119a4-kube-api-access-f64wc\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:52.705763 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:52.705246    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:53.035692 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:53.035661    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"]
Oct 30 12:48:53.037528 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:48:53.037500    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fb24330_4337_4c41_80d0_e780d94119a4.slice/crio-988f4d75b3fb841c70c10a3b6a0e699018918a1acc47fdb17605ba3d7231ad35 WatchSource:0}: Error finding container 988f4d75b3fb841c70c10a3b6a0e699018918a1acc47fdb17605ba3d7231ad35: Status 404 returned error can't find the container with id 988f4d75b3fb841c70c10a3b6a0e699018918a1acc47fdb17605ba3d7231ad35
Oct 30 12:48:53.040012 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:53.039974    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:53.040085 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:53.040057    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:53.040138 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:53.040123    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:53.190885 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:53.190843    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" event={"ID":"4fb24330-4337-4c41-80d0-e780d94119a4","Type":"ContainerStarted","Data":"52858f0b5a606ad18beff2bfc7ee74a728a5d9dbd4804574c8e480ffe1e9d9d3"}
Oct 30 12:48:53.191081 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:53.190921    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" event={"ID":"4fb24330-4337-4c41-80d0-e780d94119a4","Type":"ContainerStarted","Data":"988f4d75b3fb841c70c10a3b6a0e699018918a1acc47fdb17605ba3d7231ad35"}
Oct 30 12:48:53.249642 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:53.249566    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" podStartSLOduration=1.249550829 podStartE2EDuration="1.249550829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:48:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:48:53.246810265 +0000 UTC m=+777.824759354" watchObservedRunningTime="2026-10-30 12:48:53.249550829 +0000 UTC m=+777.827499919"
Oct 30 12:48:53.706806 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:53.706767    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:48:54.710151 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:54.710115    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 12:48:54.710813 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:54.710767    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 12:48:55.706608 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:55.706563    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 12:48:55.706770 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:55.706622    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 12:48:55.953045 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:55.953011    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"]
Oct 30 12:48:55.959265 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:55.959207    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.006162 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.006121    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"]
Oct 30 12:48:56.053110 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.053078    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-workload-certs\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.053270 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.053151    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/579ade75-eb45-4c2b-add1-d199586065c0-istio-token\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.053270 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.053250    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/579ade75-eb45-4c2b-add1-d199586065c0-istiod-ca-cert\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.053399 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.053306    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-workload-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.053455 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.053406    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j45kf\" (UniqueName: \"kubernetes.io/projected/579ade75-eb45-4c2b-add1-d199586065c0-kube-api-access-j45kf\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.053507 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.053461    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/579ade75-eb45-4c2b-add1-d199586065c0-istio-podinfo\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.053555 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.053530    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-istio-envoy\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.053661 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.053644    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-istio-data\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.053710 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.053681    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-credential-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.154326 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.154289    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-credential-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.154510 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.154371    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-workload-certs\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.154510 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.154410    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/579ade75-eb45-4c2b-add1-d199586065c0-istio-token\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.154510 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.154431    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/579ade75-eb45-4c2b-add1-d199586065c0-istiod-ca-cert\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.154510 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.154459    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-workload-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.154510 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.154504    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j45kf\" (UniqueName: \"kubernetes.io/projected/579ade75-eb45-4c2b-add1-d199586065c0-kube-api-access-j45kf\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.154827 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.154540    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/579ade75-eb45-4c2b-add1-d199586065c0-istio-podinfo\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.154827 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.154569    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-istio-envoy\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.154827 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.154666    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-istio-data\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.154962 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.154873    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-credential-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.155004 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.154950    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-workload-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.155051 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.155016    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-istio-data\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.155235 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.155212    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-workload-certs\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.155327 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.155239    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/579ade75-eb45-4c2b-add1-d199586065c0-istiod-ca-cert\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.156940 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.156921    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-istio-envoy\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.157195 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.157170    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/579ade75-eb45-4c2b-add1-d199586065c0-istio-podinfo\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.203220 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.203189    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j45kf\" (UniqueName: \"kubernetes.io/projected/579ade75-eb45-4c2b-add1-d199586065c0-kube-api-access-j45kf\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.205990 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.205951    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/579ade75-eb45-4c2b-add1-d199586065c0-istio-token\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.227565 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:48:56.227484    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:48:56.314961 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.314930    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:56.669665 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.669638    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"]
Oct 30 12:48:56.704313 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.704281    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:56.704410 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.704345    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:56.704410 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.704376    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:48:56.706358 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.706334    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 12:48:56.706460 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:56.706376    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 12:48:57.234857 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:57.234818    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt" event={"ID":"579ade75-eb45-4c2b-add1-d199586065c0","Type":"ContainerStarted","Data":"595cf7c70eabfad4fbf8790bb1bf32c8f4c9008fb8c061dcfd5aa58281924055"}
Oct 30 12:48:57.235340 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:57.234876    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt" event={"ID":"579ade75-eb45-4c2b-add1-d199586065c0","Type":"ContainerStarted","Data":"1539c76052be20aeae1bedccaa5720bd1051b32034ddfbaeb7a7ca428d2da3a9"}
Oct 30 12:48:57.320845 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:57.320806    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:48:57.323765 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:57.323734    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 12:48:57.323984 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:57.323956    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt" podUID="579ade75-eb45-4c2b-add1-d199586065c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 12:48:57.333381 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:57.333313    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt" podStartSLOduration=2.3332938 podStartE2EDuration="2.3332938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:48:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:48:57.302190337 +0000 UTC m=+781.880139449" watchObservedRunningTime="2026-10-30 12:48:57.3332938 +0000 UTC m=+781.911242890"
Oct 30 12:48:57.712622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:57.710552    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 12:48:57.712622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:57.710644    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 12:48:58.333130 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:58.323012    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 12:48:58.333130 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:58.323091    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt" podUID="579ade75-eb45-4c2b-add1-d199586065c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 12:48:58.710623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:58.710052    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 12:48:58.710623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:58.710132    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 12:48:59.318630 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:59.318116    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 12:48:59.318630 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:59.318180    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt" podUID="579ade75-eb45-4c2b-add1-d199586065c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 12:48:59.707061 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:59.707022    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 30 12:48:59.707824 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:48:59.707699    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 30 12:49:00.316624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:00.316213    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 12:49:00.316624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:00.316284    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt" podUID="579ade75-eb45-4c2b-add1-d199586065c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 12:49:00.710587 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:00.710553    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:49:00.711789 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:00.711745    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:49:00.712275 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:00.712257    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 12:49:01.315984 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:01.315948    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 12:49:01.316150 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:01.316022    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt" podUID="579ade75-eb45-4c2b-add1-d199586065c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 12:49:02.316128 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:02.316086    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 12:49:02.316545 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:02.316148    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt" podUID="579ade75-eb45-4c2b-add1-d199586065c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 12:49:03.316288 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:03.316251    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 30 12:49:03.316702 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:03.316314    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt" podUID="579ade75-eb45-4c2b-add1-d199586065c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 30 12:49:04.320263 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:04.320231    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:49:04.320779 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:04.320546    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:49:04.321166 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:04.321146    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 12:49:07.762206 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.762164    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"]
Oct 30 12:49:07.767734 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.767712    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.784576 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.784553    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-7a590a-openshift-default-dockercfg-4zgs6\""
Oct 30 12:49:07.801464 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.801432    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"]
Oct 30 12:49:07.860126 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.860088    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-token\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.860266 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.860147    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-workload-certs\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.860266 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.860231    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-workload-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.860343 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.860284    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-data\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.860343 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.860331    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istiod-ca-cert\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.860429 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.860416    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-credential-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.860466 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.860446    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-podinfo\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.860531 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.860513    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6k8k\" (UniqueName: \"kubernetes.io/projected/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-kube-api-access-h6k8k\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.860566 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.860545    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-envoy\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.961168 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.961131    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-workload-certs\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.961168 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.961174    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-workload-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.961408 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.961348    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-data\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.961521 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.961423    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istiod-ca-cert\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.961521 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.961503    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-credential-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.961669 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.961552    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-podinfo\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.961669 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.961572    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-workload-certs\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.961669 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.961588    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6k8k\" (UniqueName: \"kubernetes.io/projected/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-kube-api-access-h6k8k\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.961669 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.961646    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-workload-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.961885 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.961652    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-envoy\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.961885 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.961754    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-token\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.962008 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.961898    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-credential-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.962008 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.961672    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-data\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.962322 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.962292    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istiod-ca-cert\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.963904 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.963877    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-envoy\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.964159 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.964140    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-podinfo\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.980246 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.980218    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-token\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:07.980928 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:07.980905    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6k8k\" (UniqueName: \"kubernetes.io/projected/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-kube-api-access-h6k8k\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:08.084207 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:08.084131    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:08.400026 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:08.399999    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"]
Oct 30 12:49:08.402883 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:49:08.402853    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod492c3c1e_1d46_4e6f_9203_3d0a478fe8b3.slice/crio-0d3aa12477f7d5de3cbc955e1ca7f4a7909e2b87bc069c51fa68ced774adb50f WatchSource:0}: Error finding container 0d3aa12477f7d5de3cbc955e1ca7f4a7909e2b87bc069c51fa68ced774adb50f: Status 404 returned error can't find the container with id 0d3aa12477f7d5de3cbc955e1ca7f4a7909e2b87bc069c51fa68ced774adb50f
Oct 30 12:49:08.405895 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:08.405857    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:08.405995 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:08.405931    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:08.405995 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:08.405960    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:09.339047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:09.338962    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk" event={"ID":"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3","Type":"ContainerStarted","Data":"c0341fe52dcfb8e1d994335f8acfd2da7b613f0e3ea4711a740fb11d93216843"}
Oct 30 12:49:09.339047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:09.339033    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk" event={"ID":"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3","Type":"ContainerStarted","Data":"0d3aa12477f7d5de3cbc955e1ca7f4a7909e2b87bc069c51fa68ced774adb50f"}
Oct 30 12:49:09.425730 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:09.425669    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk" podStartSLOduration=2.425653949 podStartE2EDuration="2.425653949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:49:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:49:09.412172219 +0000 UTC m=+793.990121334" watchObservedRunningTime="2026-10-30 12:49:09.425653949 +0000 UTC m=+794.003603040"
Oct 30 12:49:10.090627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:10.088722    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:10.090627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:10.088883    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 12:49:10.090627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:10.088931    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk" podUID="492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 12:49:11.101041 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:11.101007    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 12:49:11.101734 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:11.101673    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk" podUID="492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 12:49:12.085965 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:12.085928    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 12:49:12.086253 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:12.086225    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk" podUID="492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 12:49:13.096623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:13.086989    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 12:49:13.096623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:13.087052    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk" podUID="492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 12:49:14.091625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:14.086957    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 12:49:14.091625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:14.087018    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk" podUID="492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 12:49:15.085024 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:15.084980    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Oct 30 12:49:15.085420 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:15.085068    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk" podUID="492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Oct 30 12:49:16.089035 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:16.089005    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:16.089543 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:16.089520    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:16.090457 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:16.090429    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 12:49:26.193516 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:49:26.193470    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:49:37.499449 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.499416    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"]
Oct 30 12:49:37.505376 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.505352    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.538643 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.538616    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"]
Oct 30 12:49:37.650754 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.650722    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-credential-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.650927 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.650766    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-data\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.650927 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.650864    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-workload-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.650927 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.650897    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-envoy\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.651060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.650928    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-workload-certs\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.651060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.651014    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-podinfo\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.651060 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.651050    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-token\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.651195 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.651064    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmr2r\" (UniqueName: \"kubernetes.io/projected/f3611f91-0fbc-47ff-b790-d635fb9d2e21-kube-api-access-bmr2r\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.651265 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.651203    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istiod-ca-cert\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.752349 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.752261    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istiod-ca-cert\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.752349 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.752341    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-credential-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.752573 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.752365    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-data\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.752573 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.752402    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-workload-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.752573 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.752507    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-envoy\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.752573 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.752558    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-workload-certs\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.752807 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.752588    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-podinfo\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.752807 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.752660    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-token\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.752807 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.752688    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmr2r\" (UniqueName: \"kubernetes.io/projected/f3611f91-0fbc-47ff-b790-d635fb9d2e21-kube-api-access-bmr2r\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.752958 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.752882    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-workload-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.753142 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.753115    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-data\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.753265 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.753124    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istiod-ca-cert\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.753265 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.753199    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-credential-socket\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.753265 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.753129    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-workload-certs\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.755266 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.755236    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-envoy\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.755525 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.755498    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-podinfo\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.774513 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.774487    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmr2r\" (UniqueName: \"kubernetes.io/projected/f3611f91-0fbc-47ff-b790-d635fb9d2e21-kube-api-access-bmr2r\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.775696 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.775675    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-token\") pod \"e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") " pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:37.823898 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:37.823868    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:38.139174 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:38.139150    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"]
Oct 30 12:49:38.142020 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:49:38.141985    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3611f91_0fbc_47ff_b790_d635fb9d2e21.slice/crio-d853c6bb3b95ac0c84ac6a6ca2077ad927c53334b0ffef813a7a05163474f64a WatchSource:0}: Error finding container d853c6bb3b95ac0c84ac6a6ca2077ad927c53334b0ffef813a7a05163474f64a: Status 404 returned error can't find the container with id d853c6bb3b95ac0c84ac6a6ca2077ad927c53334b0ffef813a7a05163474f64a
Oct 30 12:49:38.144544 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:38.144513    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:38.144645 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:38.144578    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:38.144645 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:38.144625    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:38.656197 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:38.656155    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" event={"ID":"f3611f91-0fbc-47ff-b790-d635fb9d2e21","Type":"ContainerStarted","Data":"48afee0b128284e6b675ee6a12e24b315a85bba4f4108e3c8c28f1634c2c553f"}
Oct 30 12:49:38.656762 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:38.656228    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" event={"ID":"f3611f91-0fbc-47ff-b790-d635fb9d2e21","Type":"ContainerStarted","Data":"d853c6bb3b95ac0c84ac6a6ca2077ad927c53334b0ffef813a7a05163474f64a"}
Oct 30 12:49:38.716357 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:38.716300    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" podStartSLOduration=1.7162837199999998 podStartE2EDuration="1.71628372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:49:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:49:38.715066096 +0000 UTC m=+823.293015187" watchObservedRunningTime="2026-10-30 12:49:38.71628372 +0000 UTC m=+823.294232808"
Oct 30 12:49:38.824979 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:38.824895    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:39.826000 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:39.825961    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 12:49:39.826391 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:39.826036    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 12:49:40.824218 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:40.824180    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 12:49:40.824389 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:40.824250    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 12:49:41.706559 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.706521    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"]
Oct 30 12:49:41.710570 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.710544    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.723886 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.723844    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/026e8b6e-3faa-4835-9815-489c17d71b82-istiod-ca-cert\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.724290 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.724126    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2x28\" (UniqueName: \"kubernetes.io/projected/026e8b6e-3faa-4835-9815-489c17d71b82-kube-api-access-z2x28\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.724462 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.724442    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-credential-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.724635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.724589    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/026e8b6e-3faa-4835-9815-489c17d71b82-istio-token\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.724828 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.724810    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-istio-envoy\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.725442 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.725053    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-istio-data\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.725442 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.725162    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-workload-certs\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.725442 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.725215    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/026e8b6e-3faa-4835-9815-489c17d71b82-istio-podinfo\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.725442 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.725261    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-workload-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.731378 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.731354    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"]
Oct 30 12:49:41.824267 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.824231    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 12:49:41.824471 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.824304    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 12:49:41.827494 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.827456    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-workload-certs\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.827714 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.827692    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/026e8b6e-3faa-4835-9815-489c17d71b82-istio-podinfo\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.827866 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.827850    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-workload-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.828032 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.828016    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/026e8b6e-3faa-4835-9815-489c17d71b82-istiod-ca-cert\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.828149 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.828135    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2x28\" (UniqueName: \"kubernetes.io/projected/026e8b6e-3faa-4835-9815-489c17d71b82-kube-api-access-z2x28\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.828258 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.828244    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-credential-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.828365 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.828352    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/026e8b6e-3faa-4835-9815-489c17d71b82-istio-token\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.829526 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.828426    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-workload-certs\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.829526 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.828473    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-istio-envoy\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.829526 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.828511    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-istio-data\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.829526 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.829041    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-istio-data\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.829526 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.829110    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/026e8b6e-3faa-4835-9815-489c17d71b82-istiod-ca-cert\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.829863 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.829642    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-credential-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.834399 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.832431    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-istio-envoy\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.834399 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.832540    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/026e8b6e-3faa-4835-9815-489c17d71b82-istio-podinfo\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.834399 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.832740    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-workload-socket\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.847726 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.847704    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/026e8b6e-3faa-4835-9815-489c17d71b82-istio-token\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:41.847894 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:41.847876    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2x28\" (UniqueName: \"kubernetes.io/projected/026e8b6e-3faa-4835-9815-489c17d71b82-kube-api-access-z2x28\") pod \"e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") " pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:42.031984 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:42.031891    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:42.401290 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:42.401253    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"]
Oct 30 12:49:42.405429 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:49:42.402160    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026e8b6e_3faa_4835_9815_489c17d71b82.slice/crio-25e116b492d818138e8988b5817c478f27ec2a05bb05cc88f5f3ef0ce27ca710 WatchSource:0}: Error finding container 25e116b492d818138e8988b5817c478f27ec2a05bb05cc88f5f3ef0ce27ca710: Status 404 returned error can't find the container with id 25e116b492d818138e8988b5817c478f27ec2a05bb05cc88f5f3ef0ce27ca710
Oct 30 12:49:42.409653 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:42.409575    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:42.409751 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:42.409698    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:42.409751 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:42.409728    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:42.709435 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:42.709286    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" event={"ID":"026e8b6e-3faa-4835-9815-489c17d71b82","Type":"ContainerStarted","Data":"b166ec4cf3d0df4978bf46edee0409ea729f486f19d82a86eb5877d51dfa5aca"}
Oct 30 12:49:42.709435 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:42.709337    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" event={"ID":"026e8b6e-3faa-4835-9815-489c17d71b82","Type":"ContainerStarted","Data":"25e116b492d818138e8988b5817c478f27ec2a05bb05cc88f5f3ef0ce27ca710"}
Oct 30 12:49:42.753363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:42.753300    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" podStartSLOduration=1.7532825669999998 podStartE2EDuration="1.753282567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:49:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:49:42.74930655 +0000 UTC m=+827.327255640" watchObservedRunningTime="2026-10-30 12:49:42.753282567 +0000 UTC m=+827.331231657"
Oct 30 12:49:42.825215 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:42.825183    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 12:49:42.825453 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:42.825238    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 12:49:43.032883 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:43.032787    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:43.034961 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:43.034934    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 12:49:43.035116 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:43.034982    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 12:49:43.829084 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:43.829048    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 12:49:43.829711 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:43.829679    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 12:49:44.033174 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:44.033140    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 12:49:44.033342 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:44.033194    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 12:49:44.824404 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:44.824358    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 30 12:49:44.824628 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:44.824408    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 30 12:49:45.033173 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:45.033132    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 12:49:45.033694 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:45.033197    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 12:49:45.830670 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:45.830636    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:45.831118 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:45.831106    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:45.831919 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:45.831897    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 12:49:46.035047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.035008    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 12:49:46.035466 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.035077    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 12:49:46.329047 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.329014    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"]
Oct 30 12:49:46.333679 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.333658    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.358159 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.358126    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"]
Oct 30 12:49:46.361796 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.361772    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"]
Oct 30 12:49:46.365894 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.365876    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.427428 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.427388    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"]
Oct 30 12:49:46.434629 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.434584    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istiod-ca-cert\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.434781 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.434673    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-data\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.434781 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.434719    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istiod-ca-cert\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.434887 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.434793    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r46dz\" (UniqueName: \"kubernetes.io/projected/b87eaba7-33de-46f3-a25d-2e86f1e12abe-kube-api-access-r46dz\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.434887 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.434817    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-credential-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.434887 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.434855    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-podinfo\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.435040 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.435007    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-workload-certs\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.435096 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.435074    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-token\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.435153 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.435121    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-workload-certs\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.435201 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.435168    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-envoy\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.435273 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.435248    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-envoy\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.435345 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.435328    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-workload-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.435384 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.435359    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-data\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.435426 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.435386    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-podinfo\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.435426 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.435419    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-token\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.435500 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.435461    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-workload-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.435542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.435494    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-credential-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.435542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.435522    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdbht\" (UniqueName: \"kubernetes.io/projected/26fbb81d-799f-47c7-97ab-c5a0e30425ac-kube-api-access-fdbht\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.536906 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.536858    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-workload-certs\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.537102 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.536922    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-token\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.537102 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.536949    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-workload-certs\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.537102 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.536968    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-envoy\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.537102 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.536997    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-envoy\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.537102 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.537045    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-workload-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.537102 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.537070    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-data\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.537102 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.537096    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-podinfo\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.537464 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.537330    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-workload-certs\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.537464 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.537336    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-workload-certs\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.537464 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.537451    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-data\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.538243 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.537659    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-workload-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.538243 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.537793    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-token\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.538243 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.537834    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-workload-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.538243 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.537865    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-credential-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.538243 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.537903    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdbht\" (UniqueName: \"kubernetes.io/projected/26fbb81d-799f-47c7-97ab-c5a0e30425ac-kube-api-access-fdbht\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.538243 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.538019    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istiod-ca-cert\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.538243 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.538071    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-data\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.538243 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.538116    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istiod-ca-cert\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.538243 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.538195    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r46dz\" (UniqueName: \"kubernetes.io/projected/b87eaba7-33de-46f3-a25d-2e86f1e12abe-kube-api-access-r46dz\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.538243 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.538224    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-credential-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.538779 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.538279    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-podinfo\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.538779 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.538364    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-credential-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.538779 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.538689    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-workload-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.538954 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.538932    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-data\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.540548 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.540012    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-envoy\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.540548 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.540099    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-envoy\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.540548 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.540505    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-credential-socket\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.540775 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.540613    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istiod-ca-cert\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.540775 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.540689    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istiod-ca-cert\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.541132 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.541111    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-podinfo\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.541259 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.541242    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-podinfo\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.556959 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.556932    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-token\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.557120 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.557092    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-token\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.557817 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.557800    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdbht\" (UniqueName: \"kubernetes.io/projected/26fbb81d-799f-47c7-97ab-c5a0e30425ac-kube-api-access-fdbht\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.558542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.558527    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r46dz\" (UniqueName: \"kubernetes.io/projected/b87eaba7-33de-46f3-a25d-2e86f1e12abe-kube-api-access-r46dz\") pod \"e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") " pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:46.648945 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.648860    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:46.727005 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:46.726967    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:47.011921 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:49:47.011883    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26fbb81d_799f_47c7_97ab_c5a0e30425ac.slice/crio-cd91ea90d4b8155b139befc1077798b56a6dd2ad459469305a102462bbcc2733 WatchSource:0}: Error finding container cd91ea90d4b8155b139befc1077798b56a6dd2ad459469305a102462bbcc2733: Status 404 returned error can't find the container with id cd91ea90d4b8155b139befc1077798b56a6dd2ad459469305a102462bbcc2733
Oct 30 12:49:47.013192 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:47.013163    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"]
Oct 30 12:49:47.014363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:47.014331    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:47.014465 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:47.014395    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:47.014465 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:47.014428    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:47.032525 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:47.032502    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 12:49:47.032649 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:47.032547    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 12:49:47.107624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:47.098451    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"]
Oct 30 12:49:47.108278 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:49:47.108194    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb87eaba7_33de_46f3_a25d_2e86f1e12abe.slice/crio-fc64f3e3ad6b96edb9937b3cb9e751f3cb16375a13b7eac0e26c1b3a79a09313 WatchSource:0}: Error finding container fc64f3e3ad6b96edb9937b3cb9e751f3cb16375a13b7eac0e26c1b3a79a09313: Status 404 returned error can't find the container with id fc64f3e3ad6b96edb9937b3cb9e751f3cb16375a13b7eac0e26c1b3a79a09313
Oct 30 12:49:47.114747 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:47.114709    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:47.114833 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:47.114795    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:47.114874 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:47.114837    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:47.795395 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:47.794455    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw" event={"ID":"b87eaba7-33de-46f3-a25d-2e86f1e12abe","Type":"ContainerStarted","Data":"942bad7daf144f602f87b78beb1b99bfeda4ebb86ae9d17a901c1dfa6ab2b6e5"}
Oct 30 12:49:47.795395 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:47.794526    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw" event={"ID":"b87eaba7-33de-46f3-a25d-2e86f1e12abe","Type":"ContainerStarted","Data":"fc64f3e3ad6b96edb9937b3cb9e751f3cb16375a13b7eac0e26c1b3a79a09313"}
Oct 30 12:49:47.805231 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:47.805137    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" event={"ID":"26fbb81d-799f-47c7-97ab-c5a0e30425ac","Type":"ContainerStarted","Data":"f64f4e18b1dae7ea1caa7e05a2cb5bdc0379308b1318b00d0ca1c6d4abbc64b7"}
Oct 30 12:49:47.805231 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:47.805190    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" event={"ID":"26fbb81d-799f-47c7-97ab-c5a0e30425ac","Type":"ContainerStarted","Data":"cd91ea90d4b8155b139befc1077798b56a6dd2ad459469305a102462bbcc2733"}
Oct 30 12:49:47.917443 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:47.917373    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw" podStartSLOduration=1.9173515129999998 podStartE2EDuration="1.917351513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:49:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:49:47.869853551 +0000 UTC m=+832.447802640" watchObservedRunningTime="2026-10-30 12:49:47.917351513 +0000 UTC m=+832.495300604"
Oct 30 12:49:48.035072 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:48.035028    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 12:49:48.035335 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:48.035308    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 12:49:48.655624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:48.652391    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:48.655624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:48.652581    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 12:49:48.655624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:48.652665    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 12:49:48.732713 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:48.732667    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:48.733085 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:48.733053    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 12:49:48.733197 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:48.733126    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw" podUID="b87eaba7-33de-46f3-a25d-2e86f1e12abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 12:49:49.040641 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:49.039019    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 12:49:49.040641 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:49.039097    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 12:49:49.668001 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:49.666728    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 12:49:49.668001 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:49.666801    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 12:49:49.731667 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:49.731142    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 12:49:49.731667 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:49.731233    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw" podUID="b87eaba7-33de-46f3-a25d-2e86f1e12abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 12:49:50.066668 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:50.066635    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:50.067881 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:50.067854    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:50.068673 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:50.068642    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 12:49:50.150061 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:50.150010    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" podStartSLOduration=4.149995083 podStartE2EDuration="4.149995083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:49:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:49:47.919696769 +0000 UTC m=+832.497645841" watchObservedRunningTime="2026-10-30 12:49:50.149995083 +0000 UTC m=+834.727944179"
Oct 30 12:49:50.650043 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:50.649998    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 12:49:50.650216 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:50.650044    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 12:49:50.728239 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:50.728165    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 12:49:50.728651 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:50.728253    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw" podUID="b87eaba7-33de-46f3-a25d-2e86f1e12abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 12:49:51.650620 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:51.649282    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 12:49:51.650620 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:51.649347    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 12:49:51.730645 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:51.730018    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 12:49:51.730645 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:51.730094    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw" podUID="b87eaba7-33de-46f3-a25d-2e86f1e12abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 12:49:52.649203 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:52.649166    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 12:49:52.649385 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:52.649226    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 12:49:52.733007 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:52.732969    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 12:49:52.733551 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:52.733520    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw" podUID="b87eaba7-33de-46f3-a25d-2e86f1e12abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 12:49:52.946177 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:52.946139    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"]
Oct 30 12:49:52.994623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:52.990143    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"]
Oct 30 12:49:52.994623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:52.990283    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.094376 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.094332    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-workload-certs\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.094660 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.094638    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b107d489-5cd0-492b-8cab-01d603040f8b-istiod-ca-cert\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.094872 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.094854    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-workload-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.095007 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.094991    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b107d489-5cd0-492b-8cab-01d603040f8b-istio-podinfo\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.095211 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.095191    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-istio-envoy\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.116650 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.095328    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-istio-data\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.116913 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.116884    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62thg\" (UniqueName: \"kubernetes.io/projected/b107d489-5cd0-492b-8cab-01d603040f8b-kube-api-access-62thg\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.117100 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.117082    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-credential-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.117234 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.117216    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b107d489-5cd0-492b-8cab-01d603040f8b-istio-token\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.230482 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.230388    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-workload-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.232736 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.232706    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b107d489-5cd0-492b-8cab-01d603040f8b-istio-podinfo\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.233016 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.232995    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-istio-envoy\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.233480 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.233460    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-istio-data\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.234206 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.234179    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62thg\" (UniqueName: \"kubernetes.io/projected/b107d489-5cd0-492b-8cab-01d603040f8b-kube-api-access-62thg\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.234772 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.234752    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-credential-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.235244 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.235225    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b107d489-5cd0-492b-8cab-01d603040f8b-istio-token\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.235677 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.235655    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-workload-certs\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.236127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.236109    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b107d489-5cd0-492b-8cab-01d603040f8b-istiod-ca-cert\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.237023 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.231139    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-workload-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.237023 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.236034    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-workload-certs\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.237023 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.233885    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-istio-data\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.237023 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.235158    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-credential-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.237437 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.237139    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b107d489-5cd0-492b-8cab-01d603040f8b-istiod-ca-cert\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.237437 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.237315    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-istio-envoy\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.237796 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.237771    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b107d489-5cd0-492b-8cab-01d603040f8b-istio-podinfo\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.252690 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.252668    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62thg\" (UniqueName: \"kubernetes.io/projected/b107d489-5cd0-492b-8cab-01d603040f8b-kube-api-access-62thg\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.253763 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.253741    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b107d489-5cd0-492b-8cab-01d603040f8b-istio-token\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.358958 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.358924    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:53.650659 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.650590    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 12:49:53.650848 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.650691    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 12:49:53.727665 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.727623    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 30 12:49:53.727842 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.727706    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw" podUID="b87eaba7-33de-46f3-a25d-2e86f1e12abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 30 12:49:53.786320 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.786290    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"]
Oct 30 12:49:53.788895 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:49:53.788860    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb107d489_5cd0_492b_8cab_01d603040f8b.slice/crio-599366c5e97d9c39d2079a82516536c743870e54bdf95394c09fdedb4559cc79 WatchSource:0}: Error finding container 599366c5e97d9c39d2079a82516536c743870e54bdf95394c09fdedb4559cc79: Status 404 returned error can't find the container with id 599366c5e97d9c39d2079a82516536c743870e54bdf95394c09fdedb4559cc79
Oct 30 12:49:53.792076 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.792034    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:53.792213 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.792125    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:53.792213 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.792170    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:53.999625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.998422    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd" event={"ID":"b107d489-5cd0-492b-8cab-01d603040f8b","Type":"ContainerStarted","Data":"63b413b7ea2f4104826307b87408e1df9b9589795c2bec913e508474c4db6863"}
Oct 30 12:49:53.999625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:53.998524    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd" event={"ID":"b107d489-5cd0-492b-8cab-01d603040f8b","Type":"ContainerStarted","Data":"599366c5e97d9c39d2079a82516536c743870e54bdf95394c09fdedb4559cc79"}
Oct 30 12:49:54.046988 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:54.046872    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd" podStartSLOduration=2.046853938 podStartE2EDuration="2.046853938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:49:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:49:54.045865481 +0000 UTC m=+838.623814572" watchObservedRunningTime="2026-10-30 12:49:54.046853938 +0000 UTC m=+838.624803029"
Oct 30 12:49:54.359674 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:54.359553    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:49:54.361588 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:54.361558    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 12:49:54.361750 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:54.361631    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd" podUID="b107d489-5cd0-492b-8cab-01d603040f8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 12:49:54.654020 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:54.653941    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:54.655028 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:54.655006    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:54.655608 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:54.655575    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 12:49:54.732332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:54.732300    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:54.732783 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:54.732757    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:54.733472 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:54.733448    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 12:49:55.360154 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:55.360114    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 12:49:55.360672 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:55.360179    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd" podUID="b107d489-5cd0-492b-8cab-01d603040f8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 12:49:56.123258 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.123224    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"]
Oct 30 12:49:56.128911 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.128888    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.160662 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.160629    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"]
Oct 30 12:49:56.217329 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.217236    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-token\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.217329 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.217315    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-credential-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.217672 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.217512    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-envoy\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.217750 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.217668    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-podinfo\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.217819 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.217751    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-data\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.217875 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.217817    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-workload-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.217875 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.217847    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9r8z\" (UniqueName: \"kubernetes.io/projected/facb034f-f30c-4290-8ad3-07b58b7163e0-kube-api-access-f9r8z\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.217979 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.217901    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-workload-certs\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.217979 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.217928    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/facb034f-f30c-4290-8ad3-07b58b7163e0-istiod-ca-cert\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.220827 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:49:56.220800    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:49:56.319013 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.318977    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-envoy\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.319216 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.319041    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-podinfo\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.319216 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.319091    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-data\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.319216 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.319141    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-workload-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.319381 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.319253    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9r8z\" (UniqueName: \"kubernetes.io/projected/facb034f-f30c-4290-8ad3-07b58b7163e0-kube-api-access-f9r8z\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.319381 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.319306    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-workload-certs\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.319381 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.319330    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/facb034f-f30c-4290-8ad3-07b58b7163e0-istiod-ca-cert\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.319548 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.319382    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-token\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.319548 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.319446    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-credential-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.319690 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.319538    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-data\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.319690 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.319558    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-workload-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.319836 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.319814    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-workload-certs\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.319912 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.319889    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-credential-socket\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.320173 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.320154    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/facb034f-f30c-4290-8ad3-07b58b7163e0-istiod-ca-cert\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.321386 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.321366    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-envoy\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.321778 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.321758    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-podinfo\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.337474 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.337451    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-token\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.337672 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.337652    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9r8z\" (UniqueName: \"kubernetes.io/projected/facb034f-f30c-4290-8ad3-07b58b7163e0-kube-api-access-f9r8z\") pod \"e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") " pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.359581 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.359558    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 12:49:56.359687 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.359621    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd" podUID="b107d489-5cd0-492b-8cab-01d603040f8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 12:49:56.445729 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.445701    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:56.804477 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.804442    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"]
Oct 30 12:49:56.807887 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:49:56.807853    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfacb034f_f30c_4290_8ad3_07b58b7163e0.slice/crio-783821871c6546f6f4205ae0df51ae0b1aed51283a2a936aa9ec5953ce88d94e WatchSource:0}: Error finding container 783821871c6546f6f4205ae0df51ae0b1aed51283a2a936aa9ec5953ce88d94e: Status 404 returned error can't find the container with id 783821871c6546f6f4205ae0df51ae0b1aed51283a2a936aa9ec5953ce88d94e
Oct 30 12:49:56.810416 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.810381    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:56.810527 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.810463    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:56.810527 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:56.810511    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:49:57.019671 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:57.019632    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr" event={"ID":"facb034f-f30c-4290-8ad3-07b58b7163e0","Type":"ContainerStarted","Data":"ce68dc9dab1c95a224b9fb561d51f0aa8dbe3f3ea2f94af8b8e1c1612a09874e"}
Oct 30 12:49:57.019863 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:57.019709    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr" event={"ID":"facb034f-f30c-4290-8ad3-07b58b7163e0","Type":"ContainerStarted","Data":"783821871c6546f6f4205ae0df51ae0b1aed51283a2a936aa9ec5953ce88d94e"}
Oct 30 12:49:57.062804 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:57.062703    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr" podStartSLOduration=1.062690305 podStartE2EDuration="1.062690305s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:49:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:49:57.059400206 +0000 UTC m=+841.637349295" watchObservedRunningTime="2026-10-30 12:49:57.062690305 +0000 UTC m=+841.640639393"
Oct 30 12:49:57.360162 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:57.360085    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 12:49:57.360162 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:57.360143    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd" podUID="b107d489-5cd0-492b-8cab-01d603040f8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 12:49:57.445959 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:57.445919    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:49:57.447375 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:57.447354    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 12:49:57.447444 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:57.447407    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr" podUID="facb034f-f30c-4290-8ad3-07b58b7163e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 12:49:58.359431 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:58.359395    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 12:49:58.359587 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:58.359446    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd" podUID="b107d489-5cd0-492b-8cab-01d603040f8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 12:49:58.446061 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:58.446029    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 12:49:58.446533 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:58.446077    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr" podUID="facb034f-f30c-4290-8ad3-07b58b7163e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 12:49:59.359711 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:59.359665    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 12:49:59.359908 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:59.359738    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd" podUID="b107d489-5cd0-492b-8cab-01d603040f8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 12:49:59.446390 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:59.446349    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 12:49:59.446839 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:49:59.446418    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr" podUID="facb034f-f30c-4290-8ad3-07b58b7163e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 12:50:00.360302 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:00.360263    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 30 12:50:00.360482 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:00.360334    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd" podUID="b107d489-5cd0-492b-8cab-01d603040f8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 30 12:50:00.446664 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:00.446630    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 12:50:00.447041 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:00.446704    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr" podUID="facb034f-f30c-4290-8ad3-07b58b7163e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 12:50:01.297452 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.297411    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"]
Oct 30 12:50:01.326235 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.326205    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"]
Oct 30 12:50:01.326544 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.326524    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.352699 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.352669    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-105228\"/\"kube-root-ca.crt\""
Oct 30 12:50:01.352846 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.352824    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-105228\"/\"default-dockercfg-q265w\""
Oct 30 12:50:01.352927 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.352788    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-105228\"/\"openshift-service-ca.crt\""
Oct 30 12:50:01.352927 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.352751    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-105228\"/\"e2e-conflict-a-28de59c9-kserve-self-signed-certs\""
Oct 30 12:50:01.369448 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.365051    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:50:01.369448 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.365658    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:50:01.369448 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.366653    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 12:50:01.435626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.435417    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-tls-certs\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.435626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.435496    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-tmp-dir\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.435626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.435548    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-home\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.435920 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.435752    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-model-cache\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.435920 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.435832    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68sbg\" (UniqueName: \"kubernetes.io/projected/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-kube-api-access-68sbg\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.435920 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.435876    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-dshm\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.446501 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.446072    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 12:50:01.446501 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.446136    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr" podUID="facb034f-f30c-4290-8ad3-07b58b7163e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 12:50:01.561675 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.561495    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-model-cache\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.561675 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.561591    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68sbg\" (UniqueName: \"kubernetes.io/projected/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-kube-api-access-68sbg\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.561675 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.561653    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-dshm\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.562273 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.561714    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-tls-certs\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.562273 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.561769    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-tmp-dir\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.562273 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.561806    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-home\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.562441 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.562270    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-home\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.592942 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.563924    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-tmp-dir\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.592942 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.570482    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-tls-certs\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.592942 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.570725    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-dshm\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.592942 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.592874    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-model-cache\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.742748 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.678307    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68sbg\" (UniqueName: \"kubernetes.io/projected/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-kube-api-access-68sbg\") pod \"e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") " pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:01.963414 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:01.963383    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:02.303669 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:02.303552    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"]
Oct 30 12:50:02.447105 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:02.447077    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 12:50:02.447230 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:02.447122    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr" podUID="facb034f-f30c-4290-8ad3-07b58b7163e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 12:50:03.120439 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:03.120331    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9" event={"ID":"db4c6296-b3cc-4c9a-b7dd-f86081bdda84","Type":"ContainerStarted","Data":"d600f350cdcde7affbc7bb6f3d17f8cd91c309b0a337279d6f689a339ad3fd8f"}
Oct 30 12:50:03.120439 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:03.120395    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9" event={"ID":"db4c6296-b3cc-4c9a-b7dd-f86081bdda84","Type":"ContainerStarted","Data":"4a6719e8a3096b9f679105cb5c3ae5357cd6760d77f23bee681770c710bec97a"}
Oct 30 12:50:03.161893 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:03.161836    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9" podStartSLOduration=2.161822458 podStartE2EDuration="2.161822458s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:50:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:50:03.160528967 +0000 UTC m=+847.738478061" watchObservedRunningTime="2026-10-30 12:50:03.161822458 +0000 UTC m=+847.739771546"
Oct 30 12:50:03.446930 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:03.446892    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 30 12:50:03.447118 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:03.446967    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr" podUID="facb034f-f30c-4290-8ad3-07b58b7163e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 30 12:50:03.964862 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:03.964830    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:04.493412 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:04.493378    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:50:04.496117 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:04.494839    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:50:04.496117 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:04.496012    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 12:50:06.113796 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:06.113756    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:50:06.114405 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:06.114371    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:50:06.681684 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:06.675971    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:50:07.006951 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:07.003722    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:50:07.659628 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:07.658714    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:50:07.991469 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:07.991420    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:50:08.232634 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.231299    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:50:08.294238 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.294127    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:50:08.296540 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.296515    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:50:08.482707 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.481974    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:50:08.553899 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.553814    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"]
Oct 30 12:50:08.670720 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.664353    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:50:08.681234 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.681199    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:50:08.681407 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.681294    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:50:08.684979 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.684949    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.726239 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.726209    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"]
Oct 30 12:50:08.836154 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.834425    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-workload-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.836154 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.834506    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-envoy\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.836154 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.834556    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-podinfo\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.836154 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.834637    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kldn2\" (UniqueName: \"kubernetes.io/projected/237869eb-18ff-4f5a-87e8-e7d8c97bd088-kube-api-access-kldn2\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.836154 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.834677    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-token\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.836154 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.834712    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-data\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.836154 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.834742    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-workload-certs\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.836154 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.834890    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-credential-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.836154 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.834967    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istiod-ca-cert\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.943646 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.942543    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istiod-ca-cert\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.943646 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.942675    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-workload-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.943646 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.942742    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-envoy\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.943646 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.942794    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-podinfo\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.943646 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.942861    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kldn2\" (UniqueName: \"kubernetes.io/projected/237869eb-18ff-4f5a-87e8-e7d8c97bd088-kube-api-access-kldn2\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.943646 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.942903    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-token\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.943646 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.942936    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-data\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.943646 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.942967    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-workload-certs\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.943646 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.943089    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-credential-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.944285 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.943679    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-credential-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.944349 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.944325    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istiod-ca-cert\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.950167 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.944587    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-workload-socket\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.950167 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.947372    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-envoy\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.966759 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.964426    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-data\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.966759 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.964991    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-workload-certs\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:08.995399 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:08.995356    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-podinfo\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:09.025589 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:09.025555    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-token\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:09.115664 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:09.113338    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kldn2\" (UniqueName: \"kubernetes.io/projected/237869eb-18ff-4f5a-87e8-e7d8c97bd088-kube-api-access-kldn2\") pod \"e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") " pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:09.130583 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:09.129226    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:50:09.253421 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:09.253389    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:09.602543 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:50:09.602507    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod237869eb_18ff_4f5a_87e8_e7d8c97bd088.slice/crio-8c554a49490cd6eb0ab42bd87faf72015b674091e9dddf36e3f8fe890fd4fedd WatchSource:0}: Error finding container 8c554a49490cd6eb0ab42bd87faf72015b674091e9dddf36e3f8fe890fd4fedd: Status 404 returned error can't find the container with id 8c554a49490cd6eb0ab42bd87faf72015b674091e9dddf36e3f8fe890fd4fedd
Oct 30 12:50:09.603316 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:09.603206    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"]
Oct 30 12:50:09.605275 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:09.605240    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:50:09.605375 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:09.605304    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:50:09.605375 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:09.605336    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:50:10.444730 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:10.444691    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9" event={"ID":"237869eb-18ff-4f5a-87e8-e7d8c97bd088","Type":"ContainerStarted","Data":"8185dd7b9cc870ecf6a1ad499fec7157df1842ea6f9aa7f81f2b79ca5388f322"}
Oct 30 12:50:10.445226 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:10.444752    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9" event={"ID":"237869eb-18ff-4f5a-87e8-e7d8c97bd088","Type":"ContainerStarted","Data":"8c554a49490cd6eb0ab42bd87faf72015b674091e9dddf36e3f8fe890fd4fedd"}
Oct 30 12:50:10.513408 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:10.513338    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9" podStartSLOduration=3.513316804 podStartE2EDuration="3.513316804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:50:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:50:10.496887447 +0000 UTC m=+855.074836540" watchObservedRunningTime="2026-10-30 12:50:10.513316804 +0000 UTC m=+855.091265896"
Oct 30 12:50:11.253841 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:11.253801    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:11.255289 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:11.255257    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 30 12:50:11.255423 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:11.255316    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9" podUID="237869eb-18ff-4f5a-87e8-e7d8c97bd088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 30 12:50:11.964195 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:11.964163    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:12.010726 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:12.010697    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:12.254224 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:12.254131    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 30 12:50:12.254224 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:12.254193    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9" podUID="237869eb-18ff-4f5a-87e8-e7d8c97bd088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 30 12:50:12.493341 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:12.493299    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:50:13.254361 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:13.254327    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 30 12:50:13.254794 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:13.254406    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9" podUID="237869eb-18ff-4f5a-87e8-e7d8c97bd088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 30 12:50:14.253935 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:14.253888    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 30 12:50:14.254259 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:14.254226    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9" podUID="237869eb-18ff-4f5a-87e8-e7d8c97bd088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 30 12:50:15.254430 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:15.254381    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 30 12:50:15.254852 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:15.254438    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9" podUID="237869eb-18ff-4f5a-87e8-e7d8c97bd088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 30 12:50:16.254695 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:16.254658    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 30 12:50:16.255074 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:16.254725    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9" podUID="237869eb-18ff-4f5a-87e8-e7d8c97bd088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 30 12:50:17.257878 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:17.257847    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:17.258370 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:17.258230    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:17.259091 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:17.259069    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 12:50:26.198711 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:50:26.198671    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:50:36.131763 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:36.131712    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:50:36.576622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:36.576524    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9" podUID="db4c6296-b3cc-4c9a-b7dd-f86081bdda84" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:50:37.789762 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:37.789714    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:50:38.400625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:38.393997    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:50:38.474779 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:38.474723    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9" podUID="db4c6296-b3cc-4c9a-b7dd-f86081bdda84" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:50:38.854760 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:38.854708    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:50:38.856017 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:38.855991    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:50:39.400516 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:39.400450    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:50:39.666179 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:39.666100    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:50:51.572551 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.572518    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"]
Oct 30 12:50:51.577449 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.577426    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.597891 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.597860    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"]
Oct 30 12:50:51.601566 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.601540    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-7a590a\"/\"distinct-w1-7a590a-kserve-self-signed-certs\""
Oct 30 12:50:51.701069 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.701030    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-dshm\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.701370 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.701339    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3bc8355-ae84-4811-935f-0046ec42375e-tls-certs\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.701610 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.701568    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-home\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.701777 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.701759    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-model-cache\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.701907 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.701881    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-tmp-dir\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.701975 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.701921    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvmzt\" (UniqueName: \"kubernetes.io/projected/f3bc8355-ae84-4811-935f-0046ec42375e-kube-api-access-dvmzt\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.805621 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.803550    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-tmp-dir\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.805621 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.803665    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvmzt\" (UniqueName: \"kubernetes.io/projected/f3bc8355-ae84-4811-935f-0046ec42375e-kube-api-access-dvmzt\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.805621 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.804407    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-tmp-dir\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.805621 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.804970    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-dshm\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.805983 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.805522    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3bc8355-ae84-4811-935f-0046ec42375e-tls-certs\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.813632 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.806043    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-home\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.813632 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.806251    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-model-cache\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.813632 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.806471    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-home\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.815664 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.815631    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-model-cache\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.816577 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.816547    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3bc8355-ae84-4811-935f-0046ec42375e-tls-certs\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.825524 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.825464    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-dshm\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.831275 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.831250    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvmzt\" (UniqueName: \"kubernetes.io/projected/f3bc8355-ae84-4811-935f-0046ec42375e-kube-api-access-dvmzt\") pod \"distinct-w1-7a590a-kserve-9d68ddd45-fb9d6\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") " pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:51.895529 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:51.894737    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:52.236153 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:52.236120    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"]
Oct 30 12:50:52.236701 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:50:52.236675    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3bc8355_ae84_4811_935f_0046ec42375e.slice/crio-a9507cd7e236d1b9d48072b536fc9f5f1e0bc582b471318f98a4e9828b2b1c4e WatchSource:0}: Error finding container a9507cd7e236d1b9d48072b536fc9f5f1e0bc582b471318f98a4e9828b2b1c4e: Status 404 returned error can't find the container with id a9507cd7e236d1b9d48072b536fc9f5f1e0bc582b471318f98a4e9828b2b1c4e
Oct 30 12:50:52.556474 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:52.556371    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" event={"ID":"f3bc8355-ae84-4811-935f-0046ec42375e","Type":"ContainerStarted","Data":"aa901b2bbc3a613c7e582cf86b81a687999655f324ad6a432c67737fefafad5b"}
Oct 30 12:50:52.556474 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:52.556426    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" event={"ID":"f3bc8355-ae84-4811-935f-0046ec42375e","Type":"ContainerStarted","Data":"a9507cd7e236d1b9d48072b536fc9f5f1e0bc582b471318f98a4e9828b2b1c4e"}
Oct 30 12:50:52.598346 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:52.598294    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podStartSLOduration=1.5982813519999999 podStartE2EDuration="1.598281352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:50:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:50:52.594329008 +0000 UTC m=+897.172278098" watchObservedRunningTime="2026-10-30 12:50:52.598281352 +0000 UTC m=+897.176230461"
Oct 30 12:50:52.896021 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:52.895932    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:50:55.686449 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:55.686419    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 12:50:55.693017 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:50:55.692989    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 12:50:56.192427 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:50:56.192384    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:51:01.504752 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:01.504702    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": context deadline exceeded"
Oct 30 12:51:01.655767 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:01.655721    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:01.724402 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:01.723721    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:01.751789 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:01.751363    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:51:01.751789 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:01.751441    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:01.764132 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:01.764024    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:51:01.764132 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:01.764095    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:01.880626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:01.877735    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:51:01.880626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:01.877810    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:01.901681 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:01.897301    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:51:02.009620 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:02.009071    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:51:02.083692 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:02.077398    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": context deadline exceeded"
Oct 30 12:51:03.210816 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:03.210783    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:51:04.912779 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:04.912715    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:05.131625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:05.130681    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:51:05.689828 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:05.689785    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9" podUID="db4c6296-b3cc-4c9a-b7dd-f86081bdda84" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:05.723569 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:05.723527    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:51:05.724278 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:05.724250    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:51:05.913828 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:05.913693    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:05.915453 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:05.915429    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:51:06.126745 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:06.126685    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:51:06.164188 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:06.164147    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:51:06.164546 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:06.164481    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:06.165651 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:06.165622    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:06.340565 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:06.340531    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:51:19.536728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:19.536691    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"]
Oct 30 12:51:26.194745 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:51:26.194703    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:51:31.190635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:31.187756    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:31.297452 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:31.297403    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:31.355748 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:31.343296    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:31.464739 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:31.464688    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9" podUID="db4c6296-b3cc-4c9a-b7dd-f86081bdda84" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:31.501767 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:31.501716    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:31.650737 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:31.650686    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:31.674971 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:31.674918    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:31.753316 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:31.753209    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:51:31.753674 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:31.753625    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:31.773640 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:31.770779    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:51:31.773640 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:31.770877    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:31.894776 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:31.894725    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:51:31.894964 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:31.894804    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:32.211186 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:32.210713    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": context deadline exceeded"
Oct 30 12:51:32.259427 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:32.259375    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:32.260469 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:32.260444    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:51:32.308876 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:32.308833    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:32.365330 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:32.365299    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:51:32.367980 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:32.365699    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:32.373625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:32.368237    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:51:32.476876 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:32.476777    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9" podUID="db4c6296-b3cc-4c9a-b7dd-f86081bdda84" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:32.478550 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:32.478526    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:51:32.657650 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:32.657265    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:51:32.674665 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:32.658901    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:51:32.687627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:32.686810    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:32.687627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:32.687413    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:51:32.900673 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:32.895770    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:33.289435 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:33.288719    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:33.328623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:33.318882    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:51:33.395951 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:33.393707    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:33.494623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:33.493946    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9" podUID="db4c6296-b3cc-4c9a-b7dd-f86081bdda84" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Oct 30 12:51:33.673477 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:33.672710    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:33.697626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:33.688682    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:33.973726 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:33.973682    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": context deadline exceeded"
Oct 30 12:51:34.291849 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:34.291733    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:34.322286 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:34.322242    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:34.399446 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:34.398319    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:51:34.497281 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:34.494852    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9" podUID="db4c6296-b3cc-4c9a-b7dd-f86081bdda84" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Oct 30 12:51:34.551061 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:34.550953    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:51:34.584167 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:34.584125    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9" podUID="db4c6296-b3cc-4c9a-b7dd-f86081bdda84" containerName="main" containerID="cri-o://d600f350cdcde7affbc7bb6f3d17f8cd91c309b0a337279d6f689a339ad3fd8f" gracePeriod=45
Oct 30 12:51:34.753628 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:34.700668    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:51:34.817186 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:34.780685    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:51:34.835077 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:34.828962    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9" podUID="db4c6296-b3cc-4c9a-b7dd-f86081bdda84" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": EOF"
Oct 30 12:51:34.877080 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:34.876197    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:51:34.890488 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:34.877724    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:51:34.925271 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:34.905444    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:51:35.006648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.003456    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:51:35.006648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.005290    4945 generic.go:356] "Generic (PLEG): container finished" podID="db4c6296-b3cc-4c9a-b7dd-f86081bdda84" containerID="d600f350cdcde7affbc7bb6f3d17f8cd91c309b0a337279d6f689a339ad3fd8f" exitCode=0
Oct 30 12:51:35.006648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.005415    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9" event={"ID":"db4c6296-b3cc-4c9a-b7dd-f86081bdda84","Type":"ContainerDied","Data":"d600f350cdcde7affbc7bb6f3d17f8cd91c309b0a337279d6f689a339ad3fd8f"}
Oct 30 12:51:35.495137 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.495104    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:51:35.568217 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.568128    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-dshm\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-dshm\") pod \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") "
Oct 30 12:51:35.568386 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.568222    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-tmp-dir\") pod \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") "
Oct 30 12:51:35.568386 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.568295    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-home\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-home\") pod \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") "
Oct 30 12:51:35.568386 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.568358    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-tls-certs\" (UniqueName: \"kubernetes.io/secret/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-tls-certs\") pod \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") "
Oct 30 12:51:35.568567 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.568531    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-home" pod "db4c6296-b3cc-4c9a-b7dd-f86081bdda84" (UID: "db4c6296-b3cc-4c9a-b7dd-f86081bdda84"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:51:35.568567 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.568517    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-tmp-dir" pod "db4c6296-b3cc-4c9a-b7dd-f86081bdda84" (UID: "db4c6296-b3cc-4c9a-b7dd-f86081bdda84"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:51:35.568774 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.568751    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-kube-api-access-68sbg\" (UniqueName: \"kubernetes.io/projected/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-kube-api-access-68sbg\") pod \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") "
Oct 30 12:51:35.568999 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.568978    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-model-cache\") pod \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\" (UID: \"db4c6296-b3cc-4c9a-b7dd-f86081bdda84\") "
Oct 30 12:51:35.569295 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.569270    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-model-cache" pod "db4c6296-b3cc-4c9a-b7dd-f86081bdda84" (UID: "db4c6296-b3cc-4c9a-b7dd-f86081bdda84"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:51:35.569626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.569591    4945 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-home\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:51:35.569718 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.569632    4945 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-model-cache\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:51:35.569718 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.569647    4945 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-tmp-dir\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:51:35.570647 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.570581    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-dshm" pod "db4c6296-b3cc-4c9a-b7dd-f86081bdda84" (UID: "db4c6296-b3cc-4c9a-b7dd-f86081bdda84"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:51:35.570739 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.570670    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-tls-certs" pod "db4c6296-b3cc-4c9a-b7dd-f86081bdda84" (UID: "db4c6296-b3cc-4c9a-b7dd-f86081bdda84"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:51:35.570795 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.570744    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-kube-api-access-68sbg" pod "db4c6296-b3cc-4c9a-b7dd-f86081bdda84" (UID: "db4c6296-b3cc-4c9a-b7dd-f86081bdda84"). InnerVolumeSpecName "kube-api-access-68sbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:51:35.670572 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.670538    4945 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-tls-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:51:35.670572 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.670570    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68sbg\" (UniqueName: \"kubernetes.io/projected/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-kube-api-access-68sbg\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:51:35.670572 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:35.670581    4945 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db4c6296-b3cc-4c9a-b7dd-f86081bdda84-dshm\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:51:36.014199 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:36.014156    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9" event={"ID":"db4c6296-b3cc-4c9a-b7dd-f86081bdda84","Type":"ContainerDied","Data":"4a6719e8a3096b9f679105cb5c3ae5357cd6760d77f23bee681770c710bec97a"}
Oct 30 12:51:36.014405 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:36.014220    4945 scope.go:122] "RemoveContainer" containerID="d600f350cdcde7affbc7bb6f3d17f8cd91c309b0a337279d6f689a339ad3fd8f"
Oct 30 12:51:36.014405 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:36.014169    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"
Oct 30 12:51:36.068490 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:36.068458    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"]
Oct 30 12:51:36.075492 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:36.075461    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9"]
Oct 30 12:51:36.462115 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:36.462072    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/e2e-conflict-a-28de59c9-kserve-f59c4c9b6-4vrr9" podUID="db4c6296-b3cc-4c9a-b7dd-f86081bdda84" containerName="main" probeResult="failure" output="Get \"https://10.132.0.78:8000/ready\": context deadline exceeded"
Oct 30 12:51:37.552024 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:37.551994    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db4c6296-b3cc-4c9a-b7dd-f86081bdda84" path="/var/lib/kubelet/pods/db4c6296-b3cc-4c9a-b7dd-f86081bdda84/volumes"
Oct 30 12:51:41.191093 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.190290    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"]
Oct 30 12:51:41.191579 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.191209    4945 cpu_manager.go:414] "containerMap: removing container" podUID="db4c6296-b3cc-4c9a-b7dd-f86081bdda84" containerName="main"
Oct 30 12:51:41.191579 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.191230    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db4c6296-b3cc-4c9a-b7dd-f86081bdda84" containerName="main"
Oct 30 12:51:41.191579 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.191413    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="db4c6296-b3cc-4c9a-b7dd-f86081bdda84" containerName="main"
Oct 30 12:51:41.195624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.195401    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.218354 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.218329    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"]
Oct 30 12:51:41.225825 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.225708    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-105228\"/\"openshift-service-ca.crt\""
Oct 30 12:51:41.225825 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.225742    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-105228\"/\"kube-root-ca.crt\""
Oct 30 12:51:41.225825 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.225779    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-105228\"/\"default-dockercfg-q265w\""
Oct 30 12:51:41.225825 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.225825    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-105228\"/\"distinct-w2-105228-kserve-self-signed-certs\""
Oct 30 12:51:41.256648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.256619    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-model-cache\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.256793 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.256761    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-home\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.256855 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.256800    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xqqm\" (UniqueName: \"kubernetes.io/projected/fbff5ead-c33b-4cce-951d-dc130152f708-kube-api-access-7xqqm\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.256910 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.256893    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-dshm\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.256955 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.256948    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-tmp-dir\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.257005 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.256973    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbff5ead-c33b-4cce-951d-dc130152f708-tls-certs\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.357767 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.357728    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-home\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.357767 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.357768    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xqqm\" (UniqueName: \"kubernetes.io/projected/fbff5ead-c33b-4cce-951d-dc130152f708-kube-api-access-7xqqm\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.358037 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.357920    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-dshm\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.358037 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.357999    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-tmp-dir\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.358148 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.358040    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbff5ead-c33b-4cce-951d-dc130152f708-tls-certs\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.358208 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.358146    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-model-cache\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.358533 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.358505    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-tmp-dir\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.358533 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.358519    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-model-cache\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.358711 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.358582    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-home\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.360309 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.360277    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-dshm\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.360664 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.360647    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbff5ead-c33b-4cce-951d-dc130152f708-tls-certs\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.377008 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.376980    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xqqm\" (UniqueName: \"kubernetes.io/projected/fbff5ead-c33b-4cce-951d-dc130152f708-kube-api-access-7xqqm\") pod \"distinct-w2-105228-kserve-5f8d848bdc-bkzmj\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") " pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.538638 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.537588    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:41.896701 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:41.896670    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"]
Oct 30 12:51:41.898552 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:51:41.898516    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbff5ead_c33b_4cce_951d_dc130152f708.slice/crio-7075f06fa09a028ac52628a5b92c79cc0bb09bc965bb683ccd7297cf522d5b9b WatchSource:0}: Error finding container 7075f06fa09a028ac52628a5b92c79cc0bb09bc965bb683ccd7297cf522d5b9b: Status 404 returned error can't find the container with id 7075f06fa09a028ac52628a5b92c79cc0bb09bc965bb683ccd7297cf522d5b9b
Oct 30 12:51:42.059027 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:42.058982    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" event={"ID":"fbff5ead-c33b-4cce-951d-dc130152f708","Type":"ContainerStarted","Data":"5926a6fbae2c0783eb6a44082f9d8a6ea6077a61e88281228d98ab99a01c6765"}
Oct 30 12:51:42.059211 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:42.059037    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" event={"ID":"fbff5ead-c33b-4cce-951d-dc130152f708","Type":"ContainerStarted","Data":"7075f06fa09a028ac52628a5b92c79cc0bb09bc965bb683ccd7297cf522d5b9b"}
Oct 30 12:51:42.102522 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:42.102465    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podStartSLOduration=1.102448124 podStartE2EDuration="1.102448124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:51:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:42.097118255 +0000 UTC m=+946.675067344" watchObservedRunningTime="2026-10-30 12:51:42.102448124 +0000 UTC m=+946.680397214"
Oct 30 12:51:42.538697 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:42.538662    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:51.539201 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:51.539165    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:51.552096 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:51.552073    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:52.142551 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:51:52.142520    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:51:56.197980 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:51:56.197934    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:52:00.141625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:00.140971    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:00.295746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:00.295696    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:00.397676 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:00.396682    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:00.696632 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:00.694697    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:00.705623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:00.703763    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:00.902625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:00.889694    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:52:01.398623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:01.313377    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:01.398623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:01.314218    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:52:01.832630 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:01.824781    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:01.958732 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:01.924715    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:52:01.958732 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:01.929283    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:52:01.958732 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:01.929650    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:02.002622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:01.959283    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:02.028650 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:02.009217    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:52:02.028650 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:02.009334    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:02.038744 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:02.034620    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:52:02.038744 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:02.034705    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:02.100925 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:02.100820    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": context deadline exceeded"
Oct 30 12:52:02.257995 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:02.257960    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:52:02.260341 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:02.260316    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:52:02.260570 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:02.260554    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:52:02.316100 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:02.316052    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:02.406780 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:02.406686    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:02.827752 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:02.827704    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:02.856237 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:02.856206    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:52:02.919721 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:02.919679    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:02.928699 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:02.928662    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:52:02.932706 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:02.932672    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:03.137764 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:03.137671    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:03.308291 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:03.308237    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:52:03.308897 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:03.308862    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9" podUID="237869eb-18ff-4f5a-87e8-e7d8c97bd088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:03.319742 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:03.319673    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:52:03.832624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:03.831805    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:03.932751 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:03.932697    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:04.012623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:04.011773    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": context deadline exceeded"
Oct 30 12:52:04.138799 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:04.138697    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:04.139843 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:04.139816    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:52:04.387625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:04.382264    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:52:04.413673 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:04.407698    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:04.844624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:04.832712    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:04.877731 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:04.877687    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:04.934797 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:04.934746    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:52:04.994632 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:04.993049    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:52:05.135336 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:05.135257    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:52:05.141623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:05.141529    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:52:05.289058 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:05.289030    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:52:16.286172 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.285723    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"]
Oct 30 12:52:16.294073 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.290930    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.305393 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.305363    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-105228\"/\"distinct-2-w2-105228-kserve-self-signed-certs\""
Oct 30 12:52:16.308232 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.308201    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"]
Oct 30 12:52:16.357660 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.356954    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-model-cache\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.357660 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.357030    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rn6w\" (UniqueName: \"kubernetes.io/projected/699038e0-b6a8-432d-96fb-bea730c1840a-kube-api-access-8rn6w\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.357660 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.357280    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-home\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.357660 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.357306    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-dshm\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.357660 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.357336    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-tmp-dir\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.357660 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.357501    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/699038e0-b6a8-432d-96fb-bea730c1840a-tls-certs\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.458962 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.458926    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/699038e0-b6a8-432d-96fb-bea730c1840a-tls-certs\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.459764 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.459736    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-model-cache\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.459989 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.459973    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rn6w\" (UniqueName: \"kubernetes.io/projected/699038e0-b6a8-432d-96fb-bea730c1840a-kube-api-access-8rn6w\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.460224 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.460197    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-model-cache\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.460328 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.460236    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-home\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.460328 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.460268    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-dshm\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.460328 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.460291    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-tmp-dir\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.460490 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.460470    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-home\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.460724 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.460703    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-tmp-dir\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.462449 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.462412    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/699038e0-b6a8-432d-96fb-bea730c1840a-tls-certs\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.463313 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.463289    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-dshm\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.497581 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.497535    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rn6w\" (UniqueName: \"kubernetes.io/projected/699038e0-b6a8-432d-96fb-bea730c1840a-kube-api-access-8rn6w\") pod \"distinct-2-w2-105228-kserve-7d7887d596-dbpp4\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") " pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.622290 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.621919    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:16.981904 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:16.981834    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"]
Oct 30 12:52:16.985047 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:52:16.985017    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod699038e0_b6a8_432d_96fb_bea730c1840a.slice/crio-7d4f2f83a876a1c7d28337e7f4cdb23515eb2f5a5d6c1deacc26284f7eccf04f WatchSource:0}: Error finding container 7d4f2f83a876a1c7d28337e7f4cdb23515eb2f5a5d6c1deacc26284f7eccf04f: Status 404 returned error can't find the container with id 7d4f2f83a876a1c7d28337e7f4cdb23515eb2f5a5d6c1deacc26284f7eccf04f
Oct 30 12:52:17.769318 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:17.769278    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" event={"ID":"699038e0-b6a8-432d-96fb-bea730c1840a","Type":"ContainerStarted","Data":"2355652ea89996073cb52d42e22aab8704aa65c36bf021a2061c5c42f882ec73"}
Oct 30 12:52:17.769847 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:17.769378    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" event={"ID":"699038e0-b6a8-432d-96fb-bea730c1840a","Type":"ContainerStarted","Data":"7d4f2f83a876a1c7d28337e7f4cdb23515eb2f5a5d6c1deacc26284f7eccf04f"}
Oct 30 12:52:17.815462 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:17.815400    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podStartSLOduration=1.815381411 podStartE2EDuration="1.815381411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:52:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:52:17.812229543 +0000 UTC m=+982.390178633" watchObservedRunningTime="2026-10-30 12:52:17.815381411 +0000 UTC m=+982.393330501"
Oct 30 12:52:18.623292 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:18.623261    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:26.199102 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:52:26.198985    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:52:26.623154 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:26.623076    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:26.635977 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:26.635950    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:26.846235 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:26.846206    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:32.848623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:32.848080    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:32.907848 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:32.907804    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:33.190805 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:33.190701    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:33.460798 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:33.460736    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:33.850989 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:33.850897    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:52:33.852065 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:33.852040    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:33.977808 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:33.977747    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": context deadline exceeded"
Oct 30 12:52:34.279201 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:34.267999    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:34.279201 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:34.269398    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:52:34.355623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:34.332862    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:34.531357 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:34.531226    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:34.566535 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:34.566504    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:52:34.837807 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:34.837686    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:34.855186 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:34.855146    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:34.872860 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:34.871690    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:52:34.979625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:34.979515    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:52:35.318564 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:35.317759    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:35.400077 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:35.400034    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:35.410634 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:35.407438    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:52:35.435733 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:35.435697    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:52:35.446726 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:35.446682    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt" podUID="579ade75-eb45-4c2b-add1-d199586065c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:35.566014 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:35.565960    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:35.614994 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:35.614847    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:52:35.615268 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:35.615203    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr" podUID="facb034f-f30c-4290-8ad3-07b58b7163e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:35.854995 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:35.854943    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:52:35.856131 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:35.856109    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:52:35.874943 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:35.874853    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:35.875527 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:35.875490    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:52:35.876065 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:35.876048    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:52:35.998624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:35.994762    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:36.014635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:36.007113    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:52:36.434798 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:36.434759    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:52:36.435079 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:36.435045    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" podUID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:36.438693 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:36.438661    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:52:36.505706 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:36.438847    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:52:36.569624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:36.567680    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:36.855743 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:36.855693    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:36.889726 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:36.889456    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:52:36.890215 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:36.889899    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:52:37.026766 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:37.026711    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:52:37.326623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:37.309933    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:52:37.507624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:37.506820    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:52:37.507624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:37.507407    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:37.749624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:37.749167    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/health\": context deadline exceeded"
Oct 30 12:52:37.850623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:37.850301    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:52:37.905396 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:37.905350    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:37.909801 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:37.906348    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:52:38.037727 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:38.037622    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:38.307907 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:38.307822    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:52:38.362750 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:38.360792    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:52:38.362750 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:38.361844    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:52:38.539432 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:38.537746    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:52:38.627710 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:38.627633    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:52:38.649200 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:52:38.649168    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:52:56.199151 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:52:56.199103    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:53:15.361208 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:15.360956    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:53:15.400857 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:15.400804    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:53:15.517546 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:15.514748    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:15.518563 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:15.517884    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:15.739966 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:15.703260    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:53:15.983839 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:15.962194    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:16.055684 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:16.048441    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:16.395148 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:16.379829    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:53:16.479624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:16.468861    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:16.491750 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:16.477489    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:53:16.491750 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:16.487949    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:53:16.558782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:16.527594    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:16.558782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:16.545181    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:53:16.628660 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:16.624708    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:16.628660 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:16.625508    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:53:16.928622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:16.922417    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:16.928622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:16.922858    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:53:17.010622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:16.992579    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:17.160625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:17.123059    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:17.160625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:17.160374    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:53:17.160625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:17.160441    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:53:17.501635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:17.493895    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:17.538038 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:17.535714    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:53:17.567796 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:17.567749    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:17.667649 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:17.667175    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:17.670655 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:17.668158    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:53:17.923027 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:17.922935    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:53:18.160838 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:18.160788    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:18.162176 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:18.160818    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:18.650711 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:18.633298    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:18.651589 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:18.651546    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:53:18.652405 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:18.652377    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:18.684713 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:18.684669    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:18.923509 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:18.923411    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:19.172732 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:19.172670    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:19.174429 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:19.174333    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:19.641757 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:19.641704    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:53:19.659014 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:19.658963    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:19.660120 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:19.659629    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:19.692731 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:19.692682    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:53:19.816030 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:19.815733    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": context deadline exceeded"
Oct 30 12:53:19.930962 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:19.930847    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:20.173615 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:20.173554    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:20.174930 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:20.174899    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:20.661635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:20.655858    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:20.687620 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:20.683676    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:20.687620 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:20.685217    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:20.755624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:20.753799    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:20.945707 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:20.945653    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:21.177626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:21.174092    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:21.177626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:21.175210    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:53:21.202902 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:21.202817    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:53:21.349140 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:21.349109    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:53:21.357332 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:21.357309    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:53:21.358258 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:21.358240    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:53:21.644630 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:21.607260    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": context deadline exceeded"
Oct 30 12:53:21.688798 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:21.687631    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:53:21.767926 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:21.767894    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:53:21.768184 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:21.768169    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:53:21.768302 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:53:21.768293    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:53:26.198353 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:53:26.198308    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:53:56.195748 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:53:56.195685    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:54:00.671721 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:00.671675    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:00.707784 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:00.707727    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:01.197625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:01.188706    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:01.197625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:01.189994    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:01.211648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:01.209188    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:01.674836 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:01.673877    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:54:01.702980 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:01.675363    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:54:01.702980 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:01.701995    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:01.917625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:01.892419    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:54:02.069851 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.069802    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:02.239968 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.210721    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:02.239968 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.211776    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:54:02.264661 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.252759    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:02.264661 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.260481    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:54:02.264661 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.260549    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:54:02.264661 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.260589    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:54:02.541950 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.541901    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:02.561679 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.553692    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:02.678739 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.678690    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:02.680439 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.680406    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:54:02.681688 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.681663    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:54:02.682999 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.682979    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:54:02.731692 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.731644    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:02.732808 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.732782    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:54:02.906231 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:02.906128    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": context deadline exceeded"
Oct 30 12:54:03.215363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:03.215317    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:54:03.216352 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:03.216321    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:03.290630 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:03.280164    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:03.701623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:03.698446    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:54:03.748773 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:03.713579    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:03.748773 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:03.714586    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:03.748773 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:03.736687    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:03.911623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:03.908870    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:54:04.013677 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:03.987698    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:04.257636 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:04.253745    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:04.348675 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:04.312852    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:54:04.353968 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:04.312768    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:04.804658 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:04.799364    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:04.804658 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:04.799839    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:04.826625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:04.808672    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:54:04.826625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:04.810148    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:05.284623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:05.268523    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:54:05.394036 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:05.364680    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:54:05.394579 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:05.364722    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:05.803798 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:05.803745    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:05.807622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:05.804043    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:05.832104 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:05.813128    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:05.833058 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:05.813164    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:54:06.171263 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.171175    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"]
Oct 30 12:54:06.311577 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.311543    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:06.355542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.355501    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"]
Oct 30 12:54:06.428706 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.426744    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:06.428706 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.427990    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:06.433746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.431823    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-metrics-certs\") pod \"maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:06.433746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.431910    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktqh2\" (UniqueName: \"kubernetes.io/projected/c8212273-cd59-49c6-9ac3-117846bcc192-kube-api-access-ktqh2\") pod \"maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:06.433746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.431982    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-maas-api-tls\") pod \"maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:06.433746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.432009    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8212273-cd59-49c6-9ac3-117846bcc192-openshift-service-ca\") pod \"maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:06.433746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.432475    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7a04e002-serving-cert\""
Oct 30 12:54:06.490623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.490453    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:06.561626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.538565    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-metrics-certs\") pod \"maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:06.561626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.538688    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktqh2\" (UniqueName: \"kubernetes.io/projected/c8212273-cd59-49c6-9ac3-117846bcc192-kube-api-access-ktqh2\") pod \"maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:06.561626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.538762    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-maas-api-tls\") pod \"maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:06.561626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.538788    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8212273-cd59-49c6-9ac3-117846bcc192-openshift-service-ca\") pod \"maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:06.561626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.539927    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8212273-cd59-49c6-9ac3-117846bcc192-openshift-service-ca\") pod \"maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:06.561626 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:54:06.542735    4945 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-serving-cert: secret "maas-api-e2e-mt-7a04e002-serving-cert" not found
Oct 30 12:54:06.561626 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:54:06.542796    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-maas-api-tls podName:c8212273-cd59-49c6-9ac3-117846bcc192 nodeName:}" failed. No retries permitted until 2026-10-30 12:54:07.04277828 +0000 UTC m=+1091.620727369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-maas-api-tls") pod "maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8" (UID: "c8212273-cd59-49c6-9ac3-117846bcc192") : secret "maas-api-e2e-mt-7a04e002-serving-cert" not found
Oct 30 12:54:06.625906 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.625873    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-metrics-certs\") pod \"maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:06.640622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.634302    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktqh2\" (UniqueName: \"kubernetes.io/projected/c8212273-cd59-49c6-9ac3-117846bcc192-kube-api-access-ktqh2\") pod \"maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:06.804492 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.804449    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:54:06.831938 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.831900    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:06.833431 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.833392    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:06.836497 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:06.836469    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:54:07.140675 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:07.140470    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-maas-api-tls\") pod \"maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:07.140849 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:54:07.140816    4945 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-serving-cert: secret "maas-api-e2e-mt-7a04e002-serving-cert" not found
Oct 30 12:54:07.140921 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:54:07.140881    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-maas-api-tls podName:c8212273-cd59-49c6-9ac3-117846bcc192 nodeName:}" failed. No retries permitted until 2026-10-30 12:54:08.140863392 +0000 UTC m=+1092.718812477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-maas-api-tls") pod "maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8" (UID: "c8212273-cd59-49c6-9ac3-117846bcc192") : secret "maas-api-e2e-mt-7a04e002-serving-cert" not found
Oct 30 12:54:07.467779 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:07.466709    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:54:07.583216 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:07.583169    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:07.586363 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:07.586329    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:07.640623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:07.634938    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:07.869705 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:07.857260    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:07.900304 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:07.900262    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:54:07.909216 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:07.901838    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:54:07.982622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:07.974125    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:08.070139 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:08.070107    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:54:08.100923 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:08.085099    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:54:08.116620 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:08.103650    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:54:08.119499 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:08.119469    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:54:08.136725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:08.132194    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:54:08.142514 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:08.142486    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:54:08.150316 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:08.150289    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:54:08.198237 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:08.198196    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-maas-api-tls\") pod \"maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:08.228746 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:08.228654    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-maas-api-tls\") pod \"maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:08.293182 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:08.293139    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:08.683745 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:08.682485    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"]
Oct 30 12:54:08.714047 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:54:08.712018    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8212273_cd59_49c6_9ac3_117846bcc192.slice/crio-7be5a65fd83cda8f5f86ed0af9f194f807d6c59c99a17eb969c9db0c5bf00e19 WatchSource:0}: Error finding container 7be5a65fd83cda8f5f86ed0af9f194f807d6c59c99a17eb969c9db0c5bf00e19: Status 404 returned error can't find the container with id 7be5a65fd83cda8f5f86ed0af9f194f807d6c59c99a17eb969c9db0c5bf00e19
Oct 30 12:54:08.717854 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:08.715048    4945 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 12:54:09.145348 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:09.145312    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8" event={"ID":"c8212273-cd59-49c6-9ac3-117846bcc192","Type":"ContainerStarted","Data":"7be5a65fd83cda8f5f86ed0af9f194f807d6c59c99a17eb969c9db0c5bf00e19"}
Oct 30 12:54:10.153651 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:10.153591    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8" event={"ID":"c8212273-cd59-49c6-9ac3-117846bcc192","Type":"ContainerStarted","Data":"c47c080ff6a523269808dcdda3827200cb6c1c4443b8d1c92b5f5026a5c72e0c"}
Oct 30 12:54:10.154059 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:10.153686    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:10.195101 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:10.195051    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8" podStartSLOduration=3.672374473 podStartE2EDuration="4.195036637s" totalImagesPullingTime="522.662164ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:54:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:54:10.192309 +0000 UTC m=+1094.770258090" watchObservedRunningTime="2026-10-30 12:54:10.195036637 +0000 UTC m=+1094.772985726"
Oct 30 12:54:16.166745 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:16.166708    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:54:19.775141 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.775102    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"]
Oct 30 12:54:19.780483 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.780460    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:19.800419 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.800372    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"]
Oct 30 12:54:19.808542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.806625    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-0a531c-serving-cert\""
Oct 30 12:54:19.862270 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.862236    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/785e97cb-c6da-4a0e-87eb-dbd5d8541753-metrics-certs\") pod \"maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:19.862449 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.862429    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/785e97cb-c6da-4a0e-87eb-dbd5d8541753-maas-api-tls\") pod \"maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:19.862562 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.862541    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65x24\" (UniqueName: \"kubernetes.io/projected/785e97cb-c6da-4a0e-87eb-dbd5d8541753-kube-api-access-65x24\") pod \"maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:19.862659 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.862589    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/785e97cb-c6da-4a0e-87eb-dbd5d8541753-openshift-service-ca\") pod \"maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:19.964028 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.963991    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/785e97cb-c6da-4a0e-87eb-dbd5d8541753-metrics-certs\") pod \"maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:19.964192 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.964119    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/785e97cb-c6da-4a0e-87eb-dbd5d8541753-maas-api-tls\") pod \"maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:19.964244 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.964190    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65x24\" (UniqueName: \"kubernetes.io/projected/785e97cb-c6da-4a0e-87eb-dbd5d8541753-kube-api-access-65x24\") pod \"maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:19.964244 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.964225    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/785e97cb-c6da-4a0e-87eb-dbd5d8541753-openshift-service-ca\") pod \"maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:19.964919 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.964892    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/785e97cb-c6da-4a0e-87eb-dbd5d8541753-openshift-service-ca\") pod \"maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:19.966388 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.966367    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/785e97cb-c6da-4a0e-87eb-dbd5d8541753-metrics-certs\") pod \"maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:19.966655 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.966634    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/785e97cb-c6da-4a0e-87eb-dbd5d8541753-maas-api-tls\") pod \"maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:19.982381 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:19.982345    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65x24\" (UniqueName: \"kubernetes.io/projected/785e97cb-c6da-4a0e-87eb-dbd5d8541753-kube-api-access-65x24\") pod \"maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:20.096171 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:20.096077    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:20.414805 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:20.414774    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"]
Oct 30 12:54:20.416107 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:54:20.416083    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod785e97cb_c6da_4a0e_87eb_dbd5d8541753.slice/crio-31bebe7059a815f6f5d484aa65f4e94550eb34d439b894867384a164b6204619 WatchSource:0}: Error finding container 31bebe7059a815f6f5d484aa65f4e94550eb34d439b894867384a164b6204619: Status 404 returned error can't find the container with id 31bebe7059a815f6f5d484aa65f4e94550eb34d439b894867384a164b6204619
Oct 30 12:54:21.252591 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:21.252557    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8" event={"ID":"785e97cb-c6da-4a0e-87eb-dbd5d8541753","Type":"ContainerStarted","Data":"9d31df66a872405cd8c94246ca3d83668e3a2a53b7c3d00960a1ff8b849718a6"}
Oct 30 12:54:21.253035 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:21.252627    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8" event={"ID":"785e97cb-c6da-4a0e-87eb-dbd5d8541753","Type":"ContainerStarted","Data":"31bebe7059a815f6f5d484aa65f4e94550eb34d439b894867384a164b6204619"}
Oct 30 12:54:21.253035 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:21.252657    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:21.299061 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:21.298988    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8" podStartSLOduration=1.9894240170000002 podStartE2EDuration="2.298967634s" totalImagesPullingTime="309.543617ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:54:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:54:21.294901339 +0000 UTC m=+1105.872850429" watchObservedRunningTime="2026-10-30 12:54:21.298967634 +0000 UTC m=+1105.876916723"
Oct 30 12:54:26.197132 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:54:26.197089    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:54:27.265223 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:27.265192    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:54:30.208350 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.208308    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"]
Oct 30 12:54:30.213940 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.213910    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.231243 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.231215    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"]
Oct 30 12:54:30.237137 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.237099    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7a04e002\"/\"openshift-service-ca.crt\""
Oct 30 12:54:30.237586 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.237565    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7a04e002\"/\"e2e-lifecycle-model-7a04e002-kserve-self-signed-certs\""
Oct 30 12:54:30.237978 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.237960    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7a04e002\"/\"kube-root-ca.crt\""
Oct 30 12:54:30.243567 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.243545    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7a04e002\"/\"default-dockercfg-8dr44\""
Oct 30 12:54:30.312684 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.312633    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-tmp-dir\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.312879 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.312789    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-dshm\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.312959 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.312935    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-home\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.313054 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.313037    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shbbl\" (UniqueName: \"kubernetes.io/projected/505509b6-733d-4056-94eb-756ffc629032-kube-api-access-shbbl\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.313152 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.313088    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/505509b6-733d-4056-94eb-756ffc629032-tls-certs\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.313152 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.313117    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-model-cache\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.414969 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.414923    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-tmp-dir\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.415267 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.415249    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-dshm\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.415504 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.415478    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-tmp-dir\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.415631 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.415582    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-home\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.415835 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.415817    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shbbl\" (UniqueName: \"kubernetes.io/projected/505509b6-733d-4056-94eb-756ffc629032-kube-api-access-shbbl\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.415956 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.415935    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-home\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.416055 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.416041    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/505509b6-733d-4056-94eb-756ffc629032-tls-certs\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.416698 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.416675    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-model-cache\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.417088 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.417048    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-model-cache\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.420937 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.420911    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-dshm\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.421703 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.421678    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/505509b6-733d-4056-94eb-756ffc629032-tls-certs\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.436118 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.436092    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shbbl\" (UniqueName: \"kubernetes.io/projected/505509b6-733d-4056-94eb-756ffc629032-kube-api-access-shbbl\") pod \"e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") " pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.532522 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.531253    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:30.946906 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:30.946857    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"]
Oct 30 12:54:30.948926 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:54:30.948891    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod505509b6_733d_4056_94eb_756ffc629032.slice/crio-f9db7eaabb815c511d192c5a7430ed4d917368fa3f5755ad9a34f1483bd08656 WatchSource:0}: Error finding container f9db7eaabb815c511d192c5a7430ed4d917368fa3f5755ad9a34f1483bd08656: Status 404 returned error can't find the container with id f9db7eaabb815c511d192c5a7430ed4d917368fa3f5755ad9a34f1483bd08656
Oct 30 12:54:31.334763 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:31.334673    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" event={"ID":"505509b6-733d-4056-94eb-756ffc629032","Type":"ContainerStarted","Data":"cdabb2129f093186645c10e691e36e6888f306281e3dbe293b8baf44d4fb3a55"}
Oct 30 12:54:31.334763 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:31.334718    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" event={"ID":"505509b6-733d-4056-94eb-756ffc629032","Type":"ContainerStarted","Data":"f9db7eaabb815c511d192c5a7430ed4d917368fa3f5755ad9a34f1483bd08656"}
Oct 30 12:54:31.374135 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:31.374080    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podStartSLOduration=1.374062328 podStartE2EDuration="1.374062328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:54:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:54:31.371546362 +0000 UTC m=+1115.949495452" watchObservedRunningTime="2026-10-30 12:54:31.374062328 +0000 UTC m=+1115.952011417"
Oct 30 12:54:31.535834 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:31.535793    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:40.535768 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:40.535721    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:40.549098 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:40.549072    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:41.502840 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:41.502795    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:49.400692 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:49.399686    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:49.616036 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:49.615967    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:54:49.749720 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:49.749675    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:49.750287 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:49.750259    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:54:49.750662 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:49.750641    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:54:49.812768 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:49.812705    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:49.813390 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:49.813030    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": context deadline exceeded"
Oct 30 12:54:50.403634 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:50.401679    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:50.405829 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:50.405772    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:50.416780 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:50.416719    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:54:50.491888 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:50.491838    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:50.617788 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:50.617738    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:50.619065 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:50.619040    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:54:50.668880 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:50.668803    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:54:50.750409 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:50.750371    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:50.750961 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:50.750652    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:50.751872 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:50.751474    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:54:50.890164 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:50.889029    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:54:50.890366 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:50.837150    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:50.891304 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:50.891053    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:54:51.415738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:51.414547    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:51.435444 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:51.416478    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:54:51.511779 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:51.511736    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:51.629625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:51.624709    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:54:51.759622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:51.752180    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:54:51.760220 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:51.760183    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:54:51.782647 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:51.779546    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:54:51.820765 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:51.814695    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:54:51.925623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:51.900028    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:54:52.118283 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:52.117932    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:52.447714 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:52.447666    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:52.550150 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:52.550103    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": context deadline exceeded"
Oct 30 12:54:52.556883 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:52.556857    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:54:52.604375 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:52.604344    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:54:52.676725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:52.676681    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:52.846372 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:52.846268    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:52.942956 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:52.939687    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:54:52.973114 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:52.973074    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:54:53.495728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:53.495679    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:53.617635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:53.606684    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:53.751679 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:53.750989    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:53.769630 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:53.769247    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:54:54.012704 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:54.009846    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:54:54.111191 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:54.111140    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:54.161637 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:54.152009    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:54:54.161637 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:54.159057    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:54:54.189622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:54.185757    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:54:54.488056 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:54.488011    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:54:54.560638 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:54.505686    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:54.561863 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:54.512373    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:54.563298 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:54.562482    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:54.611799 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:54.610734    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:54.611799 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:54.611304    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:54:54.769716 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:54.769099    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:54:54.771931 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:54.769967    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:54:54.772166 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:54.772137    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:55.177854 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:55.176431    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:55.177854 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:55.176800    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:54:55.551696 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:55.551563    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:55.575653 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:55.564025    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:55.575653 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:55.564112    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 12:54:55.659640 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:55.633765    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:55.673912 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:55.664745    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:54:55.813671 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:55.794738    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:55.813671 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:55.795110    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:54:55.813671 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:55.795615    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:54:55.813671 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:55.795659    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:55.937623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:55.931955    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:54:56.181775 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:56.181676    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:56.554824 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:56.554779    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:54:56.605790 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:54:56.605749    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:54:56.623625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:56.622157    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:54:56.660715 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:56.650086    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:54:56.675931 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:56.675889    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:56.825675 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:56.825328    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:54:56.879622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:56.868377    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:56.948220 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:56.895842    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:57.190849 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:57.190732    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:57.637814 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:57.637143    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:54:57.638343 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:57.638304    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:57.638635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:57.638407    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:57.695826 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:57.683465    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:57.695826 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:57.685749    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 12:54:57.901662 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:57.900457    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:54:57.901662 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:57.901500    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:54:57.960963 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:57.960930    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:54:57.961146 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:57.960997    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:54:57.961146 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:57.961039    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:54:57.961146 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:57.961077    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:54:57.961146 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:57.961109    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:54:57.963110 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:57.963064    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:54:57.963236 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:54:57.963128    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:55:04.786987 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:04.786951    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"]
Oct 30 12:55:04.797201 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:04.797163    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:04.820450 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:04.820403    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f9e9fd-serving-cert\""
Oct 30 12:55:04.839889 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:04.837970    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-maas-api-tls\") pod \"maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:04.839889 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:04.838033    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-openshift-service-ca\") pod \"maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:04.839889 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:04.838092    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-metrics-certs\") pod \"maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:04.839889 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:04.838124    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl52q\" (UniqueName: \"kubernetes.io/projected/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-kube-api-access-tl52q\") pod \"maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:04.839889 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:04.838871    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"]
Oct 30 12:55:04.940225 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:04.940188    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-maas-api-tls\") pod \"maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:04.940225 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:04.940232    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-openshift-service-ca\") pod \"maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:04.940501 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:04.940276    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-metrics-certs\") pod \"maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:04.940501 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:04.940296    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl52q\" (UniqueName: \"kubernetes.io/projected/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-kube-api-access-tl52q\") pod \"maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:04.940501 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:55:04.940339    4945 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-serving-cert: secret "maas-api-e2e-api-b-f9e9fd-serving-cert" not found
Oct 30 12:55:04.940501 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:55:04.940399    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-maas-api-tls podName:49bb88f8-4bcf-4a29-aa04-9c6a5b457715 nodeName:}" failed. No retries permitted until 2026-10-30 12:55:05.440384473 +0000 UTC m=+1150.018333540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-maas-api-tls") pod "maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg" (UID: "49bb88f8-4bcf-4a29-aa04-9c6a5b457715") : secret "maas-api-e2e-api-b-f9e9fd-serving-cert" not found
Oct 30 12:55:04.941070 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:04.941048    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-openshift-service-ca\") pod \"maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:04.943050 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:04.943021    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-metrics-certs\") pod \"maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:04.959737 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:04.959712    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl52q\" (UniqueName: \"kubernetes.io/projected/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-kube-api-access-tl52q\") pod \"maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:05.451481 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:05.451447    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-maas-api-tls\") pod \"maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:05.453901 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:05.453877    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-maas-api-tls\") pod \"maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:05.458818 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:05.458792    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:05.768009 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:05.767978    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"]
Oct 30 12:55:05.924664 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:05.924580    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg" event={"ID":"49bb88f8-4bcf-4a29-aa04-9c6a5b457715","Type":"ContainerStarted","Data":"426ea464706b030bea86f22b3c12d55bb1fab197740b93efdef9a4042911efa9"}
Oct 30 12:55:06.939532 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:06.939493    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg" event={"ID":"49bb88f8-4bcf-4a29-aa04-9c6a5b457715","Type":"ContainerStarted","Data":"95db6fea33c9f46baf5e061548dd924740a941f93c7b5b5dfe7570ca7526d030"}
Oct 30 12:55:06.940012 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:06.939802    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:06.982670 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:06.982586    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg" podStartSLOduration=2.50831437 podStartE2EDuration="2.982567484s" totalImagesPullingTime="474.253114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:55:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:55:06.97888185 +0000 UTC m=+1151.556830946" watchObservedRunningTime="2026-10-30 12:55:06.982567484 +0000 UTC m=+1151.560516574"
Oct 30 12:55:12.953903 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:12.953870    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:55:17.192279 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.192245    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"]
Oct 30 12:55:17.197911 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.197888    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:17.212404 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.212372    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6bda55-serving-cert\""
Oct 30 12:55:17.230006 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.229976    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"]
Oct 30 12:55:17.255522 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.255491    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-metrics-certs\") pod \"maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:17.255723 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.255701    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-maas-api-tls\") pod \"maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:17.255798 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.255743    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a37523-9500-4f43-b601-445847a475bf-openshift-service-ca\") pod \"maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:17.255855 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.255829    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2z7l\" (UniqueName: \"kubernetes.io/projected/58a37523-9500-4f43-b601-445847a475bf-kube-api-access-g2z7l\") pod \"maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:17.356712 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.356678    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a37523-9500-4f43-b601-445847a475bf-openshift-service-ca\") pod \"maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:17.356901 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.356735    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2z7l\" (UniqueName: \"kubernetes.io/projected/58a37523-9500-4f43-b601-445847a475bf-kube-api-access-g2z7l\") pod \"maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:17.356901 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.356885    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-metrics-certs\") pod \"maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:17.357078 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.357063    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-maas-api-tls\") pod \"maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:17.357227 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:55:17.357204    4945 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-serving-cert: secret "maas-api-e2e-api-a-6bda55-serving-cert" not found
Oct 30 12:55:17.357310 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:55:17.357297    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-maas-api-tls podName:58a37523-9500-4f43-b601-445847a475bf nodeName:}" failed. No retries permitted until 2026-10-30 12:55:17.85727864 +0000 UTC m=+1162.435227710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-maas-api-tls") pod "maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks" (UID: "58a37523-9500-4f43-b601-445847a475bf") : secret "maas-api-e2e-api-a-6bda55-serving-cert" not found
Oct 30 12:55:17.357396 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.357374    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a37523-9500-4f43-b601-445847a475bf-openshift-service-ca\") pod \"maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:17.359458 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.359432    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-metrics-certs\") pod \"maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:17.379277 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.379250    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2z7l\" (UniqueName: \"kubernetes.io/projected/58a37523-9500-4f43-b601-445847a475bf-kube-api-access-g2z7l\") pod \"maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:17.864397 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.864364    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-maas-api-tls\") pod \"maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:17.866911 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:17.866890    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-maas-api-tls\") pod \"maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:18.119623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:18.115720    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:18.495296 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:18.495262    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"]
Oct 30 12:55:19.035608 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:19.035569    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks" event={"ID":"58a37523-9500-4f43-b601-445847a475bf","Type":"ContainerStarted","Data":"511e34734425cfd2f676e26cffafd90947cb96c189853a5798d47229eb4a6a02"}
Oct 30 12:55:19.035795 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:19.035687    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks" event={"ID":"58a37523-9500-4f43-b601-445847a475bf","Type":"ContainerStarted","Data":"0413fba18441192f39f47616411e8406ca7748b9c0e259d4478c68bdea37ff33"}
Oct 30 12:55:19.035858 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:19.035830    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:19.088980 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:19.088924    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks" podStartSLOduration=1.7374406329999998 podStartE2EDuration="2.088910173s" totalImagesPullingTime="351.46954ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:55:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:55:19.084244056 +0000 UTC m=+1163.662193145" watchObservedRunningTime="2026-10-30 12:55:19.088910173 +0000 UTC m=+1163.666859261"
Oct 30 12:55:25.047986 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:25.047953    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:55:26.193512 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:55:26.193477    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:55:55.807413 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:55.807383    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 12:55:55.833404 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:55:55.833377    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 12:55:56.196813 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:55:56.196780    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:56:05.933816 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:05.933748    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:06.124625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:06.118720    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:56:06.124625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:06.118822    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:06.125160 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:06.125114    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:56:06.125416 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:06.125364    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:06.596623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:06.588676    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:06.596623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:06.590497    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:06.685688 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:06.681688    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:06.781773 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:06.781729    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:06.850198 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:06.849754    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:56:06.905973 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:06.905930    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:06.943153 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:06.943111    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:06.944305 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:06.944279    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:56:06.966629 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:06.960741    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:56:06.966629 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:06.960800    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" podUID="a069f35f-76b4-4583-857a-304896b3f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:07.368853 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:07.368808    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:07.708688 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:07.618886    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:07.708688 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:07.619248    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:56:07.832625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:07.712857    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:56:07.832625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:07.714542    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:07.907628 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:07.906987    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:07.943275 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:07.918433    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:56:07.943275 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:07.920560    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:56:07.945630 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:07.944233    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:08.013411 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:08.013306    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:56:08.068983 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:08.068954    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:56:08.069380 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:08.069361    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:56:08.071917 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:08.071889    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:56:08.072139 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:08.072115    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:56:08.362728 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:08.352721    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:56:08.401623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:08.391628    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:08.401623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:08.393679    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:56:08.951222 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:08.951075    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:08.974348 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:08.974297    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 12:56:08.998777 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:08.998729    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:56:09.109072 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:09.109024    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:56:09.166300 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:09.166261    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:56:09.301682 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:09.236887    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:56:09.427626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:09.287771    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:56:09.444648 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:09.433678    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:56:09.450858 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:09.328265    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:56:09.956626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:09.949115    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:56:09.956626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:09.949181    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:56:09.956626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:09.949852    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:56:09.980650 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:09.976827    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:10.011779 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:10.011731    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:56:10.130813 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:10.130770    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:56:10.443190 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:10.443082    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:56:10.512862 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:10.512829    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:56:10.839915 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:10.838013    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:56:10.839915 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:10.838079    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:11.041005 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:11.031293    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:56:11.041727 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:11.041691    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:56:11.041952 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:11.041922    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" podUID="473f256f-7ee3-4e49-b407-346e0242fcbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:11.070633 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:11.046644    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:56:11.320071 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:11.138715    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:11.350131 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:11.350083    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:11.362112 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:11.360589    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:56:11.490627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:11.489774    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:56:11.506203 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:11.505373    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:56:11.525055 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:11.524752    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:11.552591 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:11.552296    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/health\": context deadline exceeded"
Oct 30 12:56:11.993622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:11.992783    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:12.036624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:12.035677    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:56:12.068623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:12.054170    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:12.340902 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:12.340804    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:12.368974 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:12.352883    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:56:12.368974 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:12.353247    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:56:12.368974 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:12.367677    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:12.368974 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:12.368197    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:56:12.496256 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:12.495114    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:56:12.574354 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:12.550333    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:12.938624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:12.935452    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": context deadline exceeded"
Oct 30 12:56:12.993557 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:12.993512    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:12.995229 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:12.995201    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:56:13.037586 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:13.037544    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:13.368636 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:13.341083    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:13.368636 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:13.357986    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:56:13.368636 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:13.359735    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:13.368636 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:13.368541    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:13.493765 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:13.493719    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:13.495528 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:13.495478    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:13.684245 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:13.684148    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:56:13.910620 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:13.904592    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:56:13.965776 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:13.965735    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:14.011782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:13.999889    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 12:56:14.080624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:14.039448    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:56:14.258646 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:14.258536    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:56:14.358409 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:14.350026    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:56:14.376948 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:14.376923    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:56:14.378457 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:14.378433    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:56:15.168253 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:15.168219    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"]
Oct 30 12:56:15.168505 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:15.168472    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8" podUID="785e97cb-c6da-4a0e-87eb-dbd5d8541753" containerName="maas-api" containerID="cri-o://9d31df66a872405cd8c94246ca3d83668e3a2a53b7c3d00960a1ff8b849718a6" gracePeriod=30
Oct 30 12:56:15.512515 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:15.512485    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:56:15.597261 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:15.597232    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/785e97cb-c6da-4a0e-87eb-dbd5d8541753-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/785e97cb-c6da-4a0e-87eb-dbd5d8541753-openshift-service-ca\") pod \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") "
Oct 30 12:56:15.597478 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:15.597280    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/785e97cb-c6da-4a0e-87eb-dbd5d8541753-metrics-certs\" (UniqueName: \"kubernetes.io/secret/785e97cb-c6da-4a0e-87eb-dbd5d8541753-metrics-certs\") pod \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") "
Oct 30 12:56:15.597478 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:15.597314    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/785e97cb-c6da-4a0e-87eb-dbd5d8541753-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/785e97cb-c6da-4a0e-87eb-dbd5d8541753-maas-api-tls\") pod \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") "
Oct 30 12:56:15.597478 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:15.597407    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/785e97cb-c6da-4a0e-87eb-dbd5d8541753-kube-api-access-65x24\" (UniqueName: \"kubernetes.io/projected/785e97cb-c6da-4a0e-87eb-dbd5d8541753-kube-api-access-65x24\") pod \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\" (UID: \"785e97cb-c6da-4a0e-87eb-dbd5d8541753\") "
Oct 30 12:56:15.598184 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:15.598150    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/785e97cb-c6da-4a0e-87eb-dbd5d8541753-openshift-service-ca" pod "785e97cb-c6da-4a0e-87eb-dbd5d8541753" (UID: "785e97cb-c6da-4a0e-87eb-dbd5d8541753"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:56:15.599263 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:15.599237    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/785e97cb-c6da-4a0e-87eb-dbd5d8541753-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:56:15.601285 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:15.601242    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/785e97cb-c6da-4a0e-87eb-dbd5d8541753-metrics-certs" pod "785e97cb-c6da-4a0e-87eb-dbd5d8541753" (UID: "785e97cb-c6da-4a0e-87eb-dbd5d8541753"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:56:15.601437 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:15.601410    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/785e97cb-c6da-4a0e-87eb-dbd5d8541753-kube-api-access-65x24" pod "785e97cb-c6da-4a0e-87eb-dbd5d8541753" (UID: "785e97cb-c6da-4a0e-87eb-dbd5d8541753"). InnerVolumeSpecName "kube-api-access-65x24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:56:15.602478 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:15.602458    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/785e97cb-c6da-4a0e-87eb-dbd5d8541753-maas-api-tls" pod "785e97cb-c6da-4a0e-87eb-dbd5d8541753" (UID: "785e97cb-c6da-4a0e-87eb-dbd5d8541753"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:56:15.701070 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:15.700988    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/785e97cb-c6da-4a0e-87eb-dbd5d8541753-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:56:15.701070 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:15.701027    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/785e97cb-c6da-4a0e-87eb-dbd5d8541753-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:56:15.701070 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:15.701042    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65x24\" (UniqueName: \"kubernetes.io/projected/785e97cb-c6da-4a0e-87eb-dbd5d8541753-kube-api-access-65x24\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:56:16.007675 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.007594    4945 generic.go:356] "Generic (PLEG): container finished" podID="785e97cb-c6da-4a0e-87eb-dbd5d8541753" containerID="9d31df66a872405cd8c94246ca3d83668e3a2a53b7c3d00960a1ff8b849718a6" exitCode=0
Oct 30 12:56:16.008034 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.008010    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8" event={"ID":"785e97cb-c6da-4a0e-87eb-dbd5d8541753","Type":"ContainerDied","Data":"9d31df66a872405cd8c94246ca3d83668e3a2a53b7c3d00960a1ff8b849718a6"}
Oct 30 12:56:16.009832 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.008186    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8" event={"ID":"785e97cb-c6da-4a0e-87eb-dbd5d8541753","Type":"ContainerDied","Data":"31bebe7059a815f6f5d484aa65f4e94550eb34d439b894867384a164b6204619"}
Oct 30 12:56:16.009832 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.008223    4945 scope.go:122] "RemoveContainer" containerID="9d31df66a872405cd8c94246ca3d83668e3a2a53b7c3d00960a1ff8b849718a6"
Oct 30 12:56:16.009832 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.008401    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"
Oct 30 12:56:16.094543 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.093801    4945 scope.go:122] "RemoveContainer" containerID="9d31df66a872405cd8c94246ca3d83668e3a2a53b7c3d00960a1ff8b849718a6"
Oct 30 12:56:16.095983 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:56:16.095890    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d31df66a872405cd8c94246ca3d83668e3a2a53b7c3d00960a1ff8b849718a6\": container with ID starting with 9d31df66a872405cd8c94246ca3d83668e3a2a53b7c3d00960a1ff8b849718a6 not found: ID does not exist" containerID="9d31df66a872405cd8c94246ca3d83668e3a2a53b7c3d00960a1ff8b849718a6"
Oct 30 12:56:16.095983 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.095934    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d31df66a872405cd8c94246ca3d83668e3a2a53b7c3d00960a1ff8b849718a6"} err="failed to get container status \"9d31df66a872405cd8c94246ca3d83668e3a2a53b7c3d00960a1ff8b849718a6\": rpc error: code = NotFound desc = could not find container \"9d31df66a872405cd8c94246ca3d83668e3a2a53b7c3d00960a1ff8b849718a6\": container with ID starting with 9d31df66a872405cd8c94246ca3d83668e3a2a53b7c3d00960a1ff8b849718a6 not found: ID does not exist"
Oct 30 12:56:16.179516 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.179372    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"]
Oct 30 12:56:16.186812 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.186758    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-0a531c-5b4f9c5449-w8ts8"]
Oct 30 12:56:16.435592 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.435515    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"]
Oct 30 12:56:16.436536 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.436511    4945 cpu_manager.go:414] "containerMap: removing container" podUID="785e97cb-c6da-4a0e-87eb-dbd5d8541753" containerName="maas-api"
Oct 30 12:56:16.436536 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.436536    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="785e97cb-c6da-4a0e-87eb-dbd5d8541753" containerName="maas-api"
Oct 30 12:56:16.436764 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.436745    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="785e97cb-c6da-4a0e-87eb-dbd5d8541753" containerName="maas-api"
Oct 30 12:56:16.441819 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.441797    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.469077 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.469053    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"]
Oct 30 12:56:16.541206 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.541163    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7a04e002-openshift-default-dockercfg-vxgwf\""
Oct 30 12:56:16.562782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.562752    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-data\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.562950 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.562787    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-workload-certs\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.562950 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.562901    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-podinfo\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.563100 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.563072    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-envoy\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.563151 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.563129    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-token\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.563188 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.563161    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ddab487-2426-4e9f-9c72-e8151f5619cb-istiod-ca-cert\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.563233 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.563215    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-workload-socket\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.563299 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.563282    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhwrf\" (UniqueName: \"kubernetes.io/projected/6ddab487-2426-4e9f-9c72-e8151f5619cb-kube-api-access-qhwrf\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.563366 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.563351    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-credential-socket\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.664789 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.664751    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-data\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.664789 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.664793    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-workload-certs\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.665065 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.664959    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-podinfo\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.665129 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.665075    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-envoy\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.665129 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.665120    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-token\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.665230 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.665144    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ddab487-2426-4e9f-9c72-e8151f5619cb-istiod-ca-cert\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.665230 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.665212    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-workload-certs\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.665325 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.665215    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-workload-socket\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.665325 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.665251    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-data\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.665325 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.665290    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhwrf\" (UniqueName: \"kubernetes.io/projected/6ddab487-2426-4e9f-9c72-e8151f5619cb-kube-api-access-qhwrf\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.665437 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.665333    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-credential-socket\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.665533 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.665514    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-workload-socket\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.666022 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.665997    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-credential-socket\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.666420 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.666395    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ddab487-2426-4e9f-9c72-e8151f5619cb-istiod-ca-cert\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.668130 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.668106    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-envoy\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.668448 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.668425    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-podinfo\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.684778 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.684749    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-token\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.684982 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.684960    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhwrf\" (UniqueName: \"kubernetes.io/projected/6ddab487-2426-4e9f-9c72-e8151f5619cb-kube-api-access-qhwrf\") pod \"e2e-mt-7a04e002-openshift-default-6669855c89-988sj\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") " pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:16.760581 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:16.760545    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:17.090681 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:56:17.090643    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ddab487_2426_4e9f_9c72_e8151f5619cb.slice/crio-78f29e2e836420f3f23cc9f6efc74361fdc050e0a662f5e00cd30d01b6b51c2b WatchSource:0}: Error finding container 78f29e2e836420f3f23cc9f6efc74361fdc050e0a662f5e00cd30d01b6b51c2b: Status 404 returned error can't find the container with id 78f29e2e836420f3f23cc9f6efc74361fdc050e0a662f5e00cd30d01b6b51c2b
Oct 30 12:56:17.090863 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:17.090755    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"]
Oct 30 12:56:17.093377 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:17.093346    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:56:17.093469 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:17.093417    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:56:17.093469 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:17.093447    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:56:17.551025 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:17.550988    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="785e97cb-c6da-4a0e-87eb-dbd5d8541753" path="/var/lib/kubelet/pods/785e97cb-c6da-4a0e-87eb-dbd5d8541753/volumes"
Oct 30 12:56:18.028615 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:18.028572    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj" event={"ID":"6ddab487-2426-4e9f-9c72-e8151f5619cb","Type":"ContainerStarted","Data":"2342e8021585a636f7f5e7541464fe17f6adfbce78c49102eb7bcd904d12df77"}
Oct 30 12:56:18.028780 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:18.028639    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj" event={"ID":"6ddab487-2426-4e9f-9c72-e8151f5619cb","Type":"ContainerStarted","Data":"78f29e2e836420f3f23cc9f6efc74361fdc050e0a662f5e00cd30d01b6b51c2b"}
Oct 30 12:56:18.760798 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:18.760766    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:18.762467 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:18.762440    4945 patch_prober.go:29] interesting pod/e2e-mt-7a04e002-openshift-default-6669855c89-988sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 12:56:18.762636 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:18.762489    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj" podUID="6ddab487-2426-4e9f-9c72-e8151f5619cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 12:56:19.761271 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:19.761236    4945 patch_prober.go:29] interesting pod/e2e-mt-7a04e002-openshift-default-6669855c89-988sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 12:56:19.761686 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:19.761286    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj" podUID="6ddab487-2426-4e9f-9c72-e8151f5619cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 12:56:20.761776 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:20.761738    4945 patch_prober.go:29] interesting pod/e2e-mt-7a04e002-openshift-default-6669855c89-988sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 12:56:20.762168 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:20.761804    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj" podUID="6ddab487-2426-4e9f-9c72-e8151f5619cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 12:56:21.761577 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:21.761544    4945 patch_prober.go:29] interesting pod/e2e-mt-7a04e002-openshift-default-6669855c89-988sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 12:56:21.761736 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:21.761611    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj" podUID="6ddab487-2426-4e9f-9c72-e8151f5619cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 12:56:22.767399 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:22.767361    4945 patch_prober.go:29] interesting pod/e2e-mt-7a04e002-openshift-default-6669855c89-988sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Oct 30 12:56:22.768021 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:22.767986    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj" podUID="6ddab487-2426-4e9f-9c72-e8151f5619cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Oct 30 12:56:23.959136 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:23.959103    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:23.959686 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:23.959650    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:23.960572 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:23.960541    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 12:56:24.015887 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:24.015837    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj" podStartSLOduration=8.015823127 podStartE2EDuration="8.015823127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:56:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:18.092873822 +0000 UTC m=+1222.670822912" watchObservedRunningTime="2026-10-30 12:56:24.015823127 +0000 UTC m=+1228.593772215"
Oct 30 12:56:26.195511 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:56:26.195473    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:56:26.810888 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:26.810827    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"]
Oct 30 12:56:26.815299 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:26.815274    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:26.844028 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:26.843985    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 12:56:26.844198 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:26.844168    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jvqff\""
Oct 30 12:56:26.844879 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:26.844845    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 12:56:26.847729 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:26.847706    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 12:56:26.848328 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:26.848306    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"]
Oct 30 12:56:26.901191 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:26.901163    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:26.901334 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:26.901208    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:26.901375 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:26.901338    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:26.901375 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:26.901361    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:26.901466 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:26.901451    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:26.901644 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:26.901587    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:26.901703 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:26.901668    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbq28\" (UniqueName: \"kubernetes.io/projected/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-kube-api-access-sbq28\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.003434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.003401    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.003697 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.003455    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbq28\" (UniqueName: \"kubernetes.io/projected/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-kube-api-access-sbq28\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.003697 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.003534    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.003697 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.003576    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.003916 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.003704    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.003916 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.003751    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.003916 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.003846    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.004721 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.004688    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.006665 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.006615    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.006977 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.006954    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.007099 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.007078    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.007208 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.007185    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.022936 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.022914    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.023045 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.022966    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbq28\" (UniqueName: \"kubernetes.io/projected/bfbdeeba-d2f7-4b98-96f6-6136e0c79621-kube-api-access-sbq28\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6jqvg\" (UID: \"bfbdeeba-d2f7-4b98-96f6-6136e0c79621\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.130565 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.130492    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:27.448472 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:27.448444    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"]
Oct 30 12:56:27.450348 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:56:27.450317    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfbdeeba_d2f7_4b98_96f6_6136e0c79621.slice/crio-6a462786be7f5e85fe159b86f094476bc3773bc9b7252b69fa7dee76a0124225 WatchSource:0}: Error finding container 6a462786be7f5e85fe159b86f094476bc3773bc9b7252b69fa7dee76a0124225: Status 404 returned error can't find the container with id 6a462786be7f5e85fe159b86f094476bc3773bc9b7252b69fa7dee76a0124225
Oct 30 12:56:28.278789 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:28.278742    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg" event={"ID":"bfbdeeba-d2f7-4b98-96f6-6136e0c79621","Type":"ContainerStarted","Data":"6a462786be7f5e85fe159b86f094476bc3773bc9b7252b69fa7dee76a0124225"}
Oct 30 12:56:30.032613 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:30.032569    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:56:30.032872 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:30.032656    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:56:30.299816 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:30.299729    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg" event={"ID":"bfbdeeba-d2f7-4b98-96f6-6136e0c79621","Type":"ContainerStarted","Data":"8e7aba223f94015754da05e618aed665a6ae42798fc8aabfa09c2b313ae494b6"}
Oct 30 12:56:30.299979 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:30.299866    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:30.364727 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:30.364678    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg" podStartSLOduration=1.787021895 podStartE2EDuration="4.364664596s" totalImagesPullingTime="2.577642701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:56:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:30.350400437 +0000 UTC m=+1234.928349525" watchObservedRunningTime="2026-10-30 12:56:30.364664596 +0000 UTC m=+1234.942613685"
Oct 30 12:56:31.312299 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:31.312254    4945 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-6jqvg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 12:56:31.312791 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:31.312324    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg" podUID="bfbdeeba-d2f7-4b98-96f6-6136e0c79621" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 12:56:34.322798 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:56:34.322763    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6jqvg"
Oct 30 12:56:56.195969 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:56:56.195930    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:57:01.213424 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.213386    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"]
Oct 30 12:57:01.214335 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.213696    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg" podUID="49bb88f8-4bcf-4a29-aa04-9c6a5b457715" containerName="maas-api" containerID="cri-o://95db6fea33c9f46baf5e061548dd924740a941f93c7b5b5dfe7570ca7526d030" gracePeriod=30
Oct 30 12:57:01.648610 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.648289    4945 generic.go:356] "Generic (PLEG): container finished" podID="49bb88f8-4bcf-4a29-aa04-9c6a5b457715" containerID="95db6fea33c9f46baf5e061548dd924740a941f93c7b5b5dfe7570ca7526d030" exitCode=0
Oct 30 12:57:01.731996 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.731937    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg" event={"ID":"49bb88f8-4bcf-4a29-aa04-9c6a5b457715","Type":"ContainerDied","Data":"95db6fea33c9f46baf5e061548dd924740a941f93c7b5b5dfe7570ca7526d030"}
Oct 30 12:57:01.855733 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.855707    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:57:01.872901 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.872873    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-kube-api-access-tl52q\" (UniqueName: \"kubernetes.io/projected/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-kube-api-access-tl52q\") pod \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") "
Oct 30 12:57:01.873172 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.873149    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-metrics-certs\" (UniqueName: \"kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-metrics-certs\") pod \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") "
Oct 30 12:57:01.873263 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.873214    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-openshift-service-ca\") pod \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") "
Oct 30 12:57:01.873263 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.873248    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-maas-api-tls\") pod \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\" (UID: \"49bb88f8-4bcf-4a29-aa04-9c6a5b457715\") "
Oct 30 12:57:01.874344 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.874306    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-openshift-service-ca" pod "49bb88f8-4bcf-4a29-aa04-9c6a5b457715" (UID: "49bb88f8-4bcf-4a29-aa04-9c6a5b457715"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:57:01.875870 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.875840    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-maas-api-tls" pod "49bb88f8-4bcf-4a29-aa04-9c6a5b457715" (UID: "49bb88f8-4bcf-4a29-aa04-9c6a5b457715"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:57:01.876952 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.876927    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-metrics-certs" pod "49bb88f8-4bcf-4a29-aa04-9c6a5b457715" (UID: "49bb88f8-4bcf-4a29-aa04-9c6a5b457715"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:57:01.882617 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.878612    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-kube-api-access-tl52q" pod "49bb88f8-4bcf-4a29-aa04-9c6a5b457715" (UID: "49bb88f8-4bcf-4a29-aa04-9c6a5b457715"). InnerVolumeSpecName "kube-api-access-tl52q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:57:01.990992 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.990950    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:57:01.990992 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.990991    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tl52q\" (UniqueName: \"kubernetes.io/projected/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-kube-api-access-tl52q\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:57:01.991253 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.991008    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:57:01.991253 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:01.991022    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49bb88f8-4bcf-4a29-aa04-9c6a5b457715-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:57:02.658310 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:02.658277    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"
Oct 30 12:57:02.658310 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:02.658302    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg" event={"ID":"49bb88f8-4bcf-4a29-aa04-9c6a5b457715","Type":"ContainerDied","Data":"426ea464706b030bea86f22b3c12d55bb1fab197740b93efdef9a4042911efa9"}
Oct 30 12:57:02.658907 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:02.658381    4945 scope.go:122] "RemoveContainer" containerID="95db6fea33c9f46baf5e061548dd924740a941f93c7b5b5dfe7570ca7526d030"
Oct 30 12:57:02.712257 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:02.712232    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"]
Oct 30 12:57:02.723125 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:02.723097    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f9e9fd-5cf769468d-dgfrg"]
Oct 30 12:57:03.553741 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:03.553705    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49bb88f8-4bcf-4a29-aa04-9c6a5b457715" path="/var/lib/kubelet/pods/49bb88f8-4bcf-4a29-aa04-9c6a5b457715/volumes"
Oct 30 12:57:07.313020 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:07.312984    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"]
Oct 30 12:57:07.313533 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:07.313217    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks" podUID="58a37523-9500-4f43-b601-445847a475bf" containerName="maas-api" containerID="cri-o://511e34734425cfd2f676e26cffafd90947cb96c189853a5798d47229eb4a6a02" gracePeriod=30
Oct 30 12:57:07.859057 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:07.859005    4945 generic.go:356] "Generic (PLEG): container finished" podID="58a37523-9500-4f43-b601-445847a475bf" containerID="511e34734425cfd2f676e26cffafd90947cb96c189853a5798d47229eb4a6a02" exitCode=0
Oct 30 12:57:07.859467 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:07.859442    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks" event={"ID":"58a37523-9500-4f43-b601-445847a475bf","Type":"ContainerDied","Data":"511e34734425cfd2f676e26cffafd90947cb96c189853a5798d47229eb4a6a02"}
Oct 30 12:57:07.982825 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:07.982793    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:57:08.090183 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.090140    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58a37523-9500-4f43-b601-445847a475bf-kube-api-access-g2z7l\" (UniqueName: \"kubernetes.io/projected/58a37523-9500-4f43-b601-445847a475bf-kube-api-access-g2z7l\") pod \"58a37523-9500-4f43-b601-445847a475bf\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") "
Oct 30 12:57:08.090383 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.090311    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/58a37523-9500-4f43-b601-445847a475bf-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a37523-9500-4f43-b601-445847a475bf-openshift-service-ca\") pod \"58a37523-9500-4f43-b601-445847a475bf\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") "
Oct 30 12:57:08.090452 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.090435    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-metrics-certs\" (UniqueName: \"kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-metrics-certs\") pod \"58a37523-9500-4f43-b601-445847a475bf\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") "
Oct 30 12:57:08.090515 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.090453    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-maas-api-tls\") pod \"58a37523-9500-4f43-b601-445847a475bf\" (UID: \"58a37523-9500-4f43-b601-445847a475bf\") "
Oct 30 12:57:08.091286 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.091253    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58a37523-9500-4f43-b601-445847a475bf-openshift-service-ca" pod "58a37523-9500-4f43-b601-445847a475bf" (UID: "58a37523-9500-4f43-b601-445847a475bf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:57:08.094641 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.094574    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58a37523-9500-4f43-b601-445847a475bf-kube-api-access-g2z7l" pod "58a37523-9500-4f43-b601-445847a475bf" (UID: "58a37523-9500-4f43-b601-445847a475bf"). InnerVolumeSpecName "kube-api-access-g2z7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:57:08.094833 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.094800    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-maas-api-tls" pod "58a37523-9500-4f43-b601-445847a475bf" (UID: "58a37523-9500-4f43-b601-445847a475bf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:57:08.094969 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.094936    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-metrics-certs" pod "58a37523-9500-4f43-b601-445847a475bf" (UID: "58a37523-9500-4f43-b601-445847a475bf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:57:08.192705 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.192623    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a37523-9500-4f43-b601-445847a475bf-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:57:08.192705 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.192656    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:57:08.192705 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.192668    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58a37523-9500-4f43-b601-445847a475bf-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:57:08.192705 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.192680    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2z7l\" (UniqueName: \"kubernetes.io/projected/58a37523-9500-4f43-b601-445847a475bf-kube-api-access-g2z7l\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:57:08.868155 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.868119    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks" event={"ID":"58a37523-9500-4f43-b601-445847a475bf","Type":"ContainerDied","Data":"0413fba18441192f39f47616411e8406ca7748b9c0e259d4478c68bdea37ff33"}
Oct 30 12:57:08.868155 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.868143    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"
Oct 30 12:57:08.868707 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.868190    4945 scope.go:122] "RemoveContainer" containerID="511e34734425cfd2f676e26cffafd90947cb96c189853a5798d47229eb4a6a02"
Oct 30 12:57:08.932988 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.932959    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"]
Oct 30 12:57:08.944285 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:08.944259    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6bda55-869bbf94cf-c8tks"]
Oct 30 12:57:09.552475 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:09.552443    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="58a37523-9500-4f43-b601-445847a475bf" path="/var/lib/kubelet/pods/58a37523-9500-4f43-b601-445847a475bf/volumes"
Oct 30 12:57:16.188623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:16.185733    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:16.350045 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:16.350004    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:57:16.351406 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:16.350374    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:57:16.379447 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:16.379402    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:16.459233 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:16.453264    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:16.639626 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:16.635918    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:17.238635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.204813    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:17.238635 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.208058    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:57:17.258631 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.255897    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:57:17.258631 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.255954    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk" podUID="492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:17.358256 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.358194    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:57:17.360310 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.360264    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:57:17.366859 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.360570    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:57:17.366859 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.365417    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:57:17.379935 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.379903    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:17.381095 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.381070    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:17.382412 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.382394    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:57:17.421065 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.421033    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:57:17.421524 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.421305    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd" podUID="b107d489-5cd0-492b-8cab-01d603040f8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:17.472382 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.471333    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:17.496668 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.490004    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:17.496668 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.491874    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:57:17.688621 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.675665    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:17.737965 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:17.676796    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:18.205319 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:18.205266    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:57:18.234896 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:18.234863    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:57:18.235441 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:18.235420    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:57:18.374742 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:18.374694    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:57:18.405625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:18.387218    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:18.406462 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:18.406429    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:57:18.479678 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:18.464470    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:18.520768 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:18.481580    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:18.521821 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:18.521786    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:57:18.522534 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:18.522505    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:57:18.525737 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:18.525699    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:57:18.525911 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:18.525887    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9" podUID="237869eb-18ff-4f5a-87e8-e7d8c97bd088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:18.721378 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:18.719724    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:19.217621 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:19.210269    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:19.393623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:19.392729    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:19.434653 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:19.412439    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:19.513678 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:19.512721    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:19.531621 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:19.526689    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:19.531621 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:19.527094    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:57:19.725487 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:19.725437    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:19.885701 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:19.861765    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": context deadline exceeded"
Oct 30 12:57:19.887082 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:19.887032    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 12:57:20.280766 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:20.280709    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:20.394000 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:20.393916    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:57:20.417768 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:20.417723    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:20.537674 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:20.536080    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:20.537674 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:20.536380    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:20.537674 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:20.537559    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:57:20.709119 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:20.709080    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:57:20.797679 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:20.778348    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:20.895637 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:20.893706    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:20.895637 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:20.894498    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:57:21.065850 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.065766    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:57:21.117783 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.117735    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:57:21.118093 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.118042    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:21.119866 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.119752    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:57:21.120191 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.120054    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:21.312443 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.310704    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:57:21.312443 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.310784    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" podUID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:21.444703 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.418468    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:57:21.444703 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.419307    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:21.528770 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.517689    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": context deadline exceeded"
Oct 30 12:57:21.560818 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.560665    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:57:21.561006 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.560672    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/health\": context deadline exceeded"
Oct 30 12:57:21.617318 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.616743    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:21.802703 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.797694    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:21.906640 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.894697    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:21.970785 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.970725    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:57:21.970999 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:21.970792    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" podUID="a069f35f-76b4-4583-857a-304896b3f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:22.108715 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:22.104757    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:22.269734 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:22.269684    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:22.420796 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:22.420695    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:57:22.421378 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:22.421343    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:22.529881 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:22.529840    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:22.553972 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:22.553931    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:22.567853 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:22.567817    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:57:22.621775 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:22.620692    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:22.816557 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:22.816513    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:57:22.895739 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:22.895687    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:22.896995 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:22.895998    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:23.280763 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:23.280714    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:23.281415 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:23.281389    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:57:23.444046 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:23.444000    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:23.444685 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:23.444651    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:57:23.533555 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:23.533461    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:23.534618 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:23.534578    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:57:23.563619 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:23.560746    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:57:23.573791 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:23.569912    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:23.626755 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:23.626710    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:23.820758 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:23.820664    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:57:23.886457 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:23.886425    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:57:23.915550 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:23.915521    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:57:24.045620 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:24.045087    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:57:24.088545 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:24.088443    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:57:24.089429 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:24.089410    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:57:24.091542 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:24.091524    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:57:24.109374 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:24.109343    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:57:26.199629 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:57:26.199509    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:57:42.761808 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:42.761775    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"]
Oct 30 12:57:42.769265 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:42.769232    4945 cpu_manager.go:414] "containerMap: removing container" podUID="49bb88f8-4bcf-4a29-aa04-9c6a5b457715" containerName="maas-api"
Oct 30 12:57:42.771496 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:42.771475    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49bb88f8-4bcf-4a29-aa04-9c6a5b457715" containerName="maas-api"
Oct 30 12:57:42.771825 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:42.771719    4945 cpu_manager.go:414] "containerMap: removing container" podUID="58a37523-9500-4f43-b601-445847a475bf" containerName="maas-api"
Oct 30 12:57:42.772126 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:42.772084    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58a37523-9500-4f43-b601-445847a475bf" containerName="maas-api"
Oct 30 12:57:42.772530 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:42.772496    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="58a37523-9500-4f43-b601-445847a475bf" containerName="maas-api"
Oct 30 12:57:42.772745 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:42.772732    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="49bb88f8-4bcf-4a29-aa04-9c6a5b457715" containerName="maas-api"
Oct 30 12:57:42.800361 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:42.800330    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:42.855989 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:42.853836    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"]
Oct 30 12:57:42.861190 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:42.861158    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-2bfb1b-serving-cert\""
Oct 30 12:57:42.923088 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:42.922570    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6hvp\" (UniqueName: \"kubernetes.io/projected/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-kube-api-access-d6hvp\") pod \"maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:42.923088 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:42.922731    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:42.923088 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:42.922772    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:42.923088 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:42.922875    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-metrics-certs\") pod \"maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:43.028046 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:43.025435    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6hvp\" (UniqueName: \"kubernetes.io/projected/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-kube-api-access-d6hvp\") pod \"maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:43.028046 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:43.025572    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:43.028046 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:43.025647    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:43.028046 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:43.025775    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-metrics-certs\") pod \"maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:43.028046 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:57:43.026548    4945 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-serving-cert: secret "maas-api-e2e-ipp-a-2bfb1b-serving-cert" not found
Oct 30 12:57:43.028046 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:57:43.026645    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-maas-api-tls podName:e9cf7ee9-33f1-4291-834b-8289c4e7b4e0 nodeName:}" failed. No retries permitted until 2026-10-30 12:57:43.526623022 +0000 UTC m=+1308.104572106 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-maas-api-tls") pod "maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8" (UID: "e9cf7ee9-33f1-4291-834b-8289c4e7b4e0") : secret "maas-api-e2e-ipp-a-2bfb1b-serving-cert" not found
Oct 30 12:57:43.028046 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:43.027918    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:43.031884 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:43.031834    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-metrics-certs\") pod \"maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:43.055380 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:43.055345    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6hvp\" (UniqueName: \"kubernetes.io/projected/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-kube-api-access-d6hvp\") pod \"maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:43.532727 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:43.532684    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:43.535416 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:43.535387    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:43.810087 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:43.809992    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:44.166892 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:44.164567    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"]
Oct 30 12:57:44.166892 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:57:44.165571    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9cf7ee9_33f1_4291_834b_8289c4e7b4e0.slice/crio-a43aae4d62e7b9b4404595387f872a798441c0278baa39900b528ba25d426a07 WatchSource:0}: Error finding container a43aae4d62e7b9b4404595387f872a798441c0278baa39900b528ba25d426a07: Status 404 returned error can't find the container with id a43aae4d62e7b9b4404595387f872a798441c0278baa39900b528ba25d426a07
Oct 30 12:57:44.772477 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:44.772426    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8" event={"ID":"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0","Type":"ContainerStarted","Data":"3d8d9eaf6908b0980ae5760e4a9674fec60928f4d9e8a526ec59b09fde6ecc17"}
Oct 30 12:57:44.772682 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:44.772525    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8" event={"ID":"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0","Type":"ContainerStarted","Data":"a43aae4d62e7b9b4404595387f872a798441c0278baa39900b528ba25d426a07"}
Oct 30 12:57:44.772682 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:44.772572    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:44.813653 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:44.813566    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8" podStartSLOduration=2.316565722 podStartE2EDuration="2.813551754s" totalImagesPullingTime="496.986032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:57:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:44.810339831 +0000 UTC m=+1309.388288920" watchObservedRunningTime="2026-10-30 12:57:44.813551754 +0000 UTC m=+1309.391500843"
Oct 30 12:57:50.787393 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:57:50.787357    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 12:57:56.193859 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:57:56.193817    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:58:04.855479 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:04.855444    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"]
Oct 30 12:58:05.003572 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:05.002939    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8" podUID="c8212273-cd59-49c6-9ac3-117846bcc192" containerName="maas-api" containerID="cri-o://c47c080ff6a523269808dcdda3827200cb6c1c4443b8d1c92b5f5026a5c72e0c" gracePeriod=30
Oct 30 12:58:05.714820 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:05.714780    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:58:05.812887 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:05.812851    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-metrics-certs\") pod \"c8212273-cd59-49c6-9ac3-117846bcc192\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") "
Oct 30 12:58:05.813081 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:05.812922    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8212273-cd59-49c6-9ac3-117846bcc192-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8212273-cd59-49c6-9ac3-117846bcc192-openshift-service-ca\") pod \"c8212273-cd59-49c6-9ac3-117846bcc192\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") "
Oct 30 12:58:05.813081 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:05.813041    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-maas-api-tls\") pod \"c8212273-cd59-49c6-9ac3-117846bcc192\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") "
Oct 30 12:58:05.813202 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:05.813101    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8212273-cd59-49c6-9ac3-117846bcc192-kube-api-access-ktqh2\" (UniqueName: \"kubernetes.io/projected/c8212273-cd59-49c6-9ac3-117846bcc192-kube-api-access-ktqh2\") pod \"c8212273-cd59-49c6-9ac3-117846bcc192\" (UID: \"c8212273-cd59-49c6-9ac3-117846bcc192\") "
Oct 30 12:58:05.813356 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:05.813326    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8212273-cd59-49c6-9ac3-117846bcc192-openshift-service-ca" pod "c8212273-cd59-49c6-9ac3-117846bcc192" (UID: "c8212273-cd59-49c6-9ac3-117846bcc192"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 12:58:05.813948 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:05.813917    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8212273-cd59-49c6-9ac3-117846bcc192-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:58:05.815330 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:05.815272    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-maas-api-tls" pod "c8212273-cd59-49c6-9ac3-117846bcc192" (UID: "c8212273-cd59-49c6-9ac3-117846bcc192"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:05.816042 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:05.816013    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8212273-cd59-49c6-9ac3-117846bcc192-kube-api-access-ktqh2" pod "c8212273-cd59-49c6-9ac3-117846bcc192" (UID: "c8212273-cd59-49c6-9ac3-117846bcc192"). InnerVolumeSpecName "kube-api-access-ktqh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:58:05.816535 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:05.816503    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-metrics-certs" pod "c8212273-cd59-49c6-9ac3-117846bcc192" (UID: "c8212273-cd59-49c6-9ac3-117846bcc192"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:58:05.921625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:05.917323    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:58:05.921625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:05.917360    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8212273-cd59-49c6-9ac3-117846bcc192-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:58:05.921625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:05.917374    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktqh2\" (UniqueName: \"kubernetes.io/projected/c8212273-cd59-49c6-9ac3-117846bcc192-kube-api-access-ktqh2\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:58:06.028329 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:06.028290    4945 generic.go:356] "Generic (PLEG): container finished" podID="c8212273-cd59-49c6-9ac3-117846bcc192" containerID="c47c080ff6a523269808dcdda3827200cb6c1c4443b8d1c92b5f5026a5c72e0c" exitCode=0
Oct 30 12:58:06.028533 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:06.028336    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"
Oct 30 12:58:06.028533 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:06.028367    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8" event={"ID":"c8212273-cd59-49c6-9ac3-117846bcc192","Type":"ContainerDied","Data":"c47c080ff6a523269808dcdda3827200cb6c1c4443b8d1c92b5f5026a5c72e0c"}
Oct 30 12:58:06.028533 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:06.028442    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8" event={"ID":"c8212273-cd59-49c6-9ac3-117846bcc192","Type":"ContainerDied","Data":"7be5a65fd83cda8f5f86ed0af9f194f807d6c59c99a17eb969c9db0c5bf00e19"}
Oct 30 12:58:06.028533 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:06.028464    4945 scope.go:122] "RemoveContainer" containerID="c47c080ff6a523269808dcdda3827200cb6c1c4443b8d1c92b5f5026a5c72e0c"
Oct 30 12:58:06.041390 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:06.041363    4945 scope.go:122] "RemoveContainer" containerID="c47c080ff6a523269808dcdda3827200cb6c1c4443b8d1c92b5f5026a5c72e0c"
Oct 30 12:58:06.041804 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:58:06.041782    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c47c080ff6a523269808dcdda3827200cb6c1c4443b8d1c92b5f5026a5c72e0c\": container with ID starting with c47c080ff6a523269808dcdda3827200cb6c1c4443b8d1c92b5f5026a5c72e0c not found: ID does not exist" containerID="c47c080ff6a523269808dcdda3827200cb6c1c4443b8d1c92b5f5026a5c72e0c"
Oct 30 12:58:06.041898 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:06.041818    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c47c080ff6a523269808dcdda3827200cb6c1c4443b8d1c92b5f5026a5c72e0c"} err="failed to get container status \"c47c080ff6a523269808dcdda3827200cb6c1c4443b8d1c92b5f5026a5c72e0c\": rpc error: code = NotFound desc = could not find container \"c47c080ff6a523269808dcdda3827200cb6c1c4443b8d1c92b5f5026a5c72e0c\": container with ID starting with c47c080ff6a523269808dcdda3827200cb6c1c4443b8d1c92b5f5026a5c72e0c not found: ID does not exist"
Oct 30 12:58:06.090321 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:06.090269    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"]
Oct 30 12:58:06.100657 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:06.098642    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7a04e002-57d96987f6-wfjj8"]
Oct 30 12:58:07.551510 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:07.551473    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8212273-cd59-49c6-9ac3-117846bcc192" path="/var/lib/kubelet/pods/c8212273-cd59-49c6-9ac3-117846bcc192/volumes"
Oct 30 12:58:26.197102 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:58:26.197062    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:58:39.002754 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:39.002704    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:39.396761 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:39.395695    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:39.423718 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:39.422707    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:39.583614 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:39.582680    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:39.646727 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:39.646678    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:58:39.726885 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:39.726826    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:39.819524 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:39.819473    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:40.118817 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:40.118710    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:58:40.685667 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:40.119335    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 12:58:40.703956 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:40.412003    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:40.725782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:40.709204    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:58:40.725782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:40.709391    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:58:40.725782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:40.439181    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:40.725782 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:40.714770    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:40.773624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:40.644001    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:58:40.881062 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:40.753030    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:58:40.881523 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:40.644250    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:40.881861 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:40.778501    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:41.006783 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:40.882252    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:58:41.054882 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:41.054852    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:58:41.055145 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:41.055128    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:58:41.622508 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:41.622466    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:41.720173 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:41.720120    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:41.765985 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:41.720722    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:41.777668 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:41.720750    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:41.777668 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:41.721910    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 12:58:41.777668 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:41.720434    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:41.811625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:41.808577    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:58:41.873686 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:41.871744    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:58:41.873686 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:41.872146    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:58:41.922636 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:41.921725    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:41.922636 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:41.922072    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:42.160738 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:42.067192    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": context deadline exceeded"
Oct 30 12:58:42.184624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:42.107642    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:58:42.308624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:42.308353    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:58:42.785624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:42.778745    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:58:42.785624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:42.779126    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:58:42.785624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:42.779401    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:58:42.785624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:42.779667    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": context deadline exceeded"
Oct 30 12:58:42.923892 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:42.923846    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:58:42.924536 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:42.924136    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:42.987625 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:42.978701    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:43.707627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:43.704472    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:58:43.892661 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:43.838896    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:58:43.900695 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:43.900653    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:58:43.939748 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:43.901512    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:43.939748 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:43.923619    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:43.939748 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:43.924308    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:43.947865 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:43.941089    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:58:44.005677 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:43.970675    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": context deadline exceeded"
Oct 30 12:58:44.441686 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:44.435702    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:44.917795 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:44.917741    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:44.983411 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:44.923588    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:44.984249 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:44.923682    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:58:44.984759 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:44.923728    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:44.985408 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:44.924050    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:44.985916 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:44.985893    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:58:44.995493 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:44.924796    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:45.004627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:44.967428    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:45.438748 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:45.438696    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:45.442663 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:45.439543    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:58:45.994623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:45.989691    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:45.996175 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:45.996131    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 12:58:45.997087 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:45.997041    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 12:58:45.997688 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:45.997656    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 12:58:46.019279 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:45.998494    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:58:46.019279 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:46.016095    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 12:58:46.019279 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:46.016292    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 12:58:46.053624 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:45.998383    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 12:58:46.448462 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:46.448422    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 12:58:46.823637 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:46.823526    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:58:46.823891 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:46.823858    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:46.916953 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:46.916917    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 12:58:46.924715 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:46.924674    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:47.028373 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.023871    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:47.028373 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.024053    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:47.028373 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.024972    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:47.028373 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.026813    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:47.029236 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.029128    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:47.029469 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.029190    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:47.053631 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.052743    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:47.457524 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.457479    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 12:58:47.512159 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.512128    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 12:58:47.512915 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.512891    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 12:58:47.549622 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.543787    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 12:58:47.641325 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.641294    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 12:58:47.641629 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.641610    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 12:58:47.641808 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.641796    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 12:58:47.655550 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.653836    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 12:58:47.734950 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:47.734831    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:58:54.644224 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:54.644176    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7145-f91f9a"]
Oct 30 12:58:54.645999 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:54.645955    4945 cpu_manager.go:414] "containerMap: removing container" podUID="c8212273-cd59-49c6-9ac3-117846bcc192" containerName="maas-api"
Oct 30 12:58:54.646238 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:54.646215    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8212273-cd59-49c6-9ac3-117846bcc192" containerName="maas-api"
Oct 30 12:58:54.646535 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:54.646520    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8212273-cd59-49c6-9ac3-117846bcc192" containerName="maas-api"
Oct 30 12:58:54.676627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:54.651635    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7145-f91f9a"
Oct 30 12:58:54.676627 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:54.673012    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7kfp9\""
Oct 30 12:58:54.677471 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:54.677436    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7145-f91f9a"]
Oct 30 12:58:54.825304 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:54.825266    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j96ng\" (UniqueName: \"kubernetes.io/projected/bf1d1031-c860-4c58-a494-514fd846ef4b-kube-api-access-j96ng\") pod \"test-curl-7145-f91f9a\" (UID: \"bf1d1031-c860-4c58-a494-514fd846ef4b\") " pod="opendatahub/test-curl-7145-f91f9a"
Oct 30 12:58:54.927474 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:54.927372    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j96ng\" (UniqueName: \"kubernetes.io/projected/bf1d1031-c860-4c58-a494-514fd846ef4b-kube-api-access-j96ng\") pod \"test-curl-7145-f91f9a\" (UID: \"bf1d1031-c860-4c58-a494-514fd846ef4b\") " pod="opendatahub/test-curl-7145-f91f9a"
Oct 30 12:58:54.946143 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:54.946107    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j96ng\" (UniqueName: \"kubernetes.io/projected/bf1d1031-c860-4c58-a494-514fd846ef4b-kube-api-access-j96ng\") pod \"test-curl-7145-f91f9a\" (UID: \"bf1d1031-c860-4c58-a494-514fd846ef4b\") " pod="opendatahub/test-curl-7145-f91f9a"
Oct 30 12:58:54.996202 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:54.996165    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7145-f91f9a"
Oct 30 12:58:55.299703 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:55.299672    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7145-f91f9a"]
Oct 30 12:58:55.301513 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:58:55.301479    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf1d1031_c860_4c58_a494_514fd846ef4b.slice/crio-017a9ca5e2098bd63029803b4ffe7016c2979da29d2f8ea6256d93b94e7c5c60 WatchSource:0}: Error finding container 017a9ca5e2098bd63029803b4ffe7016c2979da29d2f8ea6256d93b94e7c5c60: Status 404 returned error can't find the container with id 017a9ca5e2098bd63029803b4ffe7016c2979da29d2f8ea6256d93b94e7c5c60
Oct 30 12:58:55.443980 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:55.443120    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7145-f91f9a" event={"ID":"bf1d1031-c860-4c58-a494-514fd846ef4b","Type":"ContainerStarted","Data":"017a9ca5e2098bd63029803b4ffe7016c2979da29d2f8ea6256d93b94e7c5c60"}
Oct 30 12:58:56.208407 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:58:56.208242    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:58:56.452087 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:56.452044    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7145-f91f9a" event={"ID":"bf1d1031-c860-4c58-a494-514fd846ef4b","Type":"ContainerStarted","Data":"b384023bfcdd89ece58a629b24801c7f118c21124f9105f23acceb5c11226d92"}
Oct 30 12:58:56.490301 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:56.490244    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7145-f91f9a" podStartSLOduration=1.604435404 podStartE2EDuration="2.490228305s" totalImagesPullingTime="885.792901ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:58:56.488025721 +0000 UTC m=+1381.065974812" watchObservedRunningTime="2026-10-30 12:58:56.490228305 +0000 UTC m=+1381.068177438"
Oct 30 12:58:57.309468 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:57.309432    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7145-f91f9a"]
Oct 30 12:58:57.315411 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:57.315382    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7145-f91f9a"]
Oct 30 12:58:58.382058 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.382017    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"]
Oct 30 12:58:58.382973 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.382951    4945 cpu_manager.go:414] "containerMap: removing container" podUID="bf1d1031-c860-4c58-a494-514fd846ef4b" containerName="test-curl-7145-f91f9a"
Oct 30 12:58:58.383082 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.382974    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf1d1031-c860-4c58-a494-514fd846ef4b" containerName="test-curl-7145-f91f9a"
Oct 30 12:58:58.383182 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.383164    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf1d1031-c860-4c58-a494-514fd846ef4b" containerName="test-curl-7145-f91f9a"
Oct 30 12:58:58.387651 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.387632    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.424391 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.424358    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"]
Oct 30 12:58:58.481392 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.481322    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7145-f91f9a" podUID="bf1d1031-c860-4c58-a494-514fd846ef4b" containerName="test-curl-7145-f91f9a" containerID="cri-o://b384023bfcdd89ece58a629b24801c7f118c21124f9105f23acceb5c11226d92" gracePeriod=2
Oct 30 12:58:58.482940 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.482912    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bef9404-f32d-4093-a4d2-f63a966df637-istio-podinfo\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.483027 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.482975    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-workload-certs\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.483088 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.483066    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-istio-envoy\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.483144 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.483122    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5756\" (UniqueName: \"kubernetes.io/projected/2bef9404-f32d-4093-a4d2-f63a966df637-kube-api-access-b5756\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.483288 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.483269    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-workload-socket\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.483397 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.483369    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-credential-socket\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.483530 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.483427    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bef9404-f32d-4093-a4d2-f63a966df637-istiod-ca-cert\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.483673 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.483651    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bef9404-f32d-4093-a4d2-f63a966df637-istio-token\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.483805 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.483789    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-istio-data\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.490434 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.490415    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-2bfb1b-openshift-default-dockercfg-tgvhs\""
Oct 30 12:58:58.585689 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.585659    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bef9404-f32d-4093-a4d2-f63a966df637-istio-token\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.585837 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.585733    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-istio-data\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.585999 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.585976    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bef9404-f32d-4093-a4d2-f63a966df637-istio-podinfo\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.586097 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.586053    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-workload-certs\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.586097 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.586096    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-istio-envoy\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.586303 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.586140    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5756\" (UniqueName: \"kubernetes.io/projected/2bef9404-f32d-4093-a4d2-f63a966df637-kube-api-access-b5756\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.586303 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.586172    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-workload-socket\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.586303 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.586179    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-istio-data\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.586303 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.586257    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-credential-socket\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.586520 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.586321    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bef9404-f32d-4093-a4d2-f63a966df637-istiod-ca-cert\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.586989 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.586958    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-credential-socket\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.587119 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.587097    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-workload-socket\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.587308 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.587288    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-workload-certs\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.587392 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.587341    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bef9404-f32d-4093-a4d2-f63a966df637-istiod-ca-cert\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.588816 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.588789    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-istio-envoy\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.588917 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.588895    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bef9404-f32d-4093-a4d2-f63a966df637-istio-podinfo\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.608703 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.608672    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5756\" (UniqueName: \"kubernetes.io/projected/2bef9404-f32d-4093-a4d2-f63a966df637-kube-api-access-b5756\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.608914 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.608894    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bef9404-f32d-4093-a4d2-f63a966df637-istio-token\") pod \"e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") " pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:58.719783 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:58.719746    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:58:59.063279 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:59.063255    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"]
Oct 30 12:58:59.064656 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:58:59.064581    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bef9404_f32d_4093_a4d2_f63a966df637.slice/crio-be1c9edb234ff7fc3274fdedb2c593b565ba33af6d4dc2667a685504ad70ac6b WatchSource:0}: Error finding container be1c9edb234ff7fc3274fdedb2c593b565ba33af6d4dc2667a685504ad70ac6b: Status 404 returned error can't find the container with id be1c9edb234ff7fc3274fdedb2c593b565ba33af6d4dc2667a685504ad70ac6b
Oct 30 12:58:59.067447 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:59.067414    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:58:59.067536 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:59.067478    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:58:59.067536 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:59.067506    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 12:58:59.498371 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:59.498138    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m" event={"ID":"2bef9404-f32d-4093-a4d2-f63a966df637","Type":"ContainerStarted","Data":"5145345ea68150b357904a5f2f1b906ce1bd99c0e2c8a316f5900be0bd0662b2"}
Oct 30 12:58:59.498371 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:59.498220    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m" event={"ID":"2bef9404-f32d-4093-a4d2-f63a966df637","Type":"ContainerStarted","Data":"be1c9edb234ff7fc3274fdedb2c593b565ba33af6d4dc2667a685504ad70ac6b"}
Oct 30 12:58:59.543029 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:59.542009    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m" podStartSLOduration=1.541995402 podStartE2EDuration="1.541995402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:58:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:58:59.53827701 +0000 UTC m=+1384.116226099" watchObservedRunningTime="2026-10-30 12:58:59.541995402 +0000 UTC m=+1384.119944490"
Oct 30 12:58:59.720575 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:58:59.720542    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:59:00.725623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:00.725527    4945 patch_prober.go:29] interesting pod/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 12:59:00.725623 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:00.725582    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m" podUID="2bef9404-f32d-4093-a4d2-f63a966df637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 12:59:00.860759 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:00.860736    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7145-f91f9a"
Oct 30 12:59:00.935876 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:00.935846    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf1d1031-c860-4c58-a494-514fd846ef4b-kube-api-access-j96ng\" (UniqueName: \"kubernetes.io/projected/bf1d1031-c860-4c58-a494-514fd846ef4b-kube-api-access-j96ng\") pod \"bf1d1031-c860-4c58-a494-514fd846ef4b\" (UID: \"bf1d1031-c860-4c58-a494-514fd846ef4b\") "
Oct 30 12:59:00.938642 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:00.938608    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf1d1031-c860-4c58-a494-514fd846ef4b-kube-api-access-j96ng" pod "bf1d1031-c860-4c58-a494-514fd846ef4b" (UID: "bf1d1031-c860-4c58-a494-514fd846ef4b"). InnerVolumeSpecName "kube-api-access-j96ng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:59:01.041835 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:01.041796    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j96ng\" (UniqueName: \"kubernetes.io/projected/bf1d1031-c860-4c58-a494-514fd846ef4b-kube-api-access-j96ng\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:59:01.552251 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:01.552214    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf1d1031-c860-4c58-a494-514fd846ef4b" path="/var/lib/kubelet/pods/bf1d1031-c860-4c58-a494-514fd846ef4b/volumes"
Oct 30 12:59:01.562590 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:01.562558    4945 generic.go:356] "Generic (PLEG): container finished" podID="bf1d1031-c860-4c58-a494-514fd846ef4b" containerID="b384023bfcdd89ece58a629b24801c7f118c21124f9105f23acceb5c11226d92" exitCode=137
Oct 30 12:59:01.562739 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:01.562639    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7145-f91f9a"
Oct 30 12:59:01.562739 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:01.562659    4945 scope.go:122] "RemoveContainer" containerID="b384023bfcdd89ece58a629b24801c7f118c21124f9105f23acceb5c11226d92"
Oct 30 12:59:01.588667 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:01.588641    4945 scope.go:122] "RemoveContainer" containerID="b384023bfcdd89ece58a629b24801c7f118c21124f9105f23acceb5c11226d92"
Oct 30 12:59:01.589196 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:59:01.589171    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b384023bfcdd89ece58a629b24801c7f118c21124f9105f23acceb5c11226d92\": container with ID starting with b384023bfcdd89ece58a629b24801c7f118c21124f9105f23acceb5c11226d92 not found: ID does not exist" containerID="b384023bfcdd89ece58a629b24801c7f118c21124f9105f23acceb5c11226d92"
Oct 30 12:59:01.589336 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:01.589308    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b384023bfcdd89ece58a629b24801c7f118c21124f9105f23acceb5c11226d92"} err="failed to get container status \"b384023bfcdd89ece58a629b24801c7f118c21124f9105f23acceb5c11226d92\": rpc error: code = NotFound desc = could not find container \"b384023bfcdd89ece58a629b24801c7f118c21124f9105f23acceb5c11226d92\": container with ID starting with b384023bfcdd89ece58a629b24801c7f118c21124f9105f23acceb5c11226d92 not found: ID does not exist"
Oct 30 12:59:01.735696 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:01.727066    4945 patch_prober.go:29] interesting pod/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 12:59:01.735696 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:01.727149    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m" podUID="2bef9404-f32d-4093-a4d2-f63a966df637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 12:59:02.720856 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:02.720823    4945 patch_prober.go:29] interesting pod/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 12:59:02.721039 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:02.720872    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m" podUID="2bef9404-f32d-4093-a4d2-f63a966df637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 12:59:03.400092 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.400051    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"]
Oct 30 12:59:03.407419 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.407390    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:03.413849 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.413821    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"]
Oct 30 12:59:03.429645 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.426432    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-f0a01c-serving-cert\""
Oct 30 12:59:03.510775 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.510726    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtfqp\" (UniqueName: \"kubernetes.io/projected/f2d5f437-5553-460a-b1f8-dc47da09a5a7-kube-api-access-jtfqp\") pod \"maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:03.510944 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.510798    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2d5f437-5553-460a-b1f8-dc47da09a5a7-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:03.511059 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.511037    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2d5f437-5553-460a-b1f8-dc47da09a5a7-metrics-certs\") pod \"maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:03.511143 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.511125    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2d5f437-5553-460a-b1f8-dc47da09a5a7-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:03.612305 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.612269    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtfqp\" (UniqueName: \"kubernetes.io/projected/f2d5f437-5553-460a-b1f8-dc47da09a5a7-kube-api-access-jtfqp\") pod \"maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:03.612305 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.612308    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2d5f437-5553-460a-b1f8-dc47da09a5a7-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:03.612506 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.612437    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2d5f437-5553-460a-b1f8-dc47da09a5a7-metrics-certs\") pod \"maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:03.612506 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.612476    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2d5f437-5553-460a-b1f8-dc47da09a5a7-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:03.613181 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.613156    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2d5f437-5553-460a-b1f8-dc47da09a5a7-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:03.614949 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.614927    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2d5f437-5553-460a-b1f8-dc47da09a5a7-metrics-certs\") pod \"maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:03.615028 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.614971    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2d5f437-5553-460a-b1f8-dc47da09a5a7-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:03.638567 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.638541    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtfqp\" (UniqueName: \"kubernetes.io/projected/f2d5f437-5553-460a-b1f8-dc47da09a5a7-kube-api-access-jtfqp\") pod \"maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:03.726104 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.724847    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:03.726104 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.725728    4945 patch_prober.go:29] interesting pod/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 12:59:03.726104 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:03.725793    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m" podUID="2bef9404-f32d-4093-a4d2-f63a966df637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 12:59:04.100641 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:04.100611    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"]
Oct 30 12:59:04.102819 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:59:04.102791    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2d5f437_5553_460a_b1f8_dc47da09a5a7.slice/crio-b131fd4b37d3c9757c0129e49d82fdba8a1adc98b20d674ac8b3bd908ccbb44e WatchSource:0}: Error finding container b131fd4b37d3c9757c0129e49d82fdba8a1adc98b20d674ac8b3bd908ccbb44e: Status 404 returned error can't find the container with id b131fd4b37d3c9757c0129e49d82fdba8a1adc98b20d674ac8b3bd908ccbb44e
Oct 30 12:59:04.620725 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:04.619962    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h" event={"ID":"f2d5f437-5553-460a-b1f8-dc47da09a5a7","Type":"ContainerStarted","Data":"b131fd4b37d3c9757c0129e49d82fdba8a1adc98b20d674ac8b3bd908ccbb44e"}
Oct 30 12:59:04.723664 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:04.720663    4945 patch_prober.go:29] interesting pod/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 12:59:04.723664 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:04.720733    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m" podUID="2bef9404-f32d-4093-a4d2-f63a966df637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 12:59:05.631480 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:05.631408    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h" event={"ID":"f2d5f437-5553-460a-b1f8-dc47da09a5a7","Type":"ContainerStarted","Data":"7426de4d078058cfc25d2e143c89ecfb79715bb7b043b6b68c7c592d00d95871"}
Oct 30 12:59:05.632227 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:05.631512    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:05.670420 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:05.669931    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h" podStartSLOduration=2.218713066 podStartE2EDuration="2.669912125s" totalImagesPullingTime="451.199059ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:59:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:59:05.669561178 +0000 UTC m=+1390.247510269" watchObservedRunningTime="2026-10-30 12:59:05.669912125 +0000 UTC m=+1390.247861216"
Oct 30 12:59:05.720381 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:05.720344    4945 patch_prober.go:29] interesting pod/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body=
Oct 30 12:59:05.720581 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:05.720400    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m" podUID="2bef9404-f32d-4093-a4d2-f63a966df637" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused"
Oct 30 12:59:06.724883 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:06.724800    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:59:06.725317 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:06.725124    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:59:06.725947 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:06.725927    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 12:59:11.644815 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:11.644757    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 12:59:16.295781 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.295743    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"]
Oct 30 12:59:16.300214 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.300190    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:16.319125 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.319097    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"]
Oct 30 12:59:16.325630 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.325567    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-f1222e-serving-cert\""
Oct 30 12:59:16.348211 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.348186    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-metrics-certs\") pod \"maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:16.348341 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.348252    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58nmd\" (UniqueName: \"kubernetes.io/projected/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-kube-api-access-58nmd\") pod \"maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:16.348527 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.348499    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:16.348584 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.348558    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:16.449726 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.449685    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-metrics-certs\") pod \"maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:16.449899 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.449789    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58nmd\" (UniqueName: \"kubernetes.io/projected/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-kube-api-access-58nmd\") pod \"maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:16.449998 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.449983    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:16.450036 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.450025    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:16.450813 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.450785    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:16.452712 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.452684    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-metrics-certs\") pod \"maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:16.452845 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.452814    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:16.469495 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.469468    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58nmd\" (UniqueName: \"kubernetes.io/projected/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-kube-api-access-58nmd\") pod \"maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:16.613753 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.613661    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:16.935040 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.935009    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"]
Oct 30 12:59:16.937219 ip-10-0-142-36 kubenswrapper[4945]: W1030 12:59:16.937190    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda39c4f07_8275_4aff_8cb5_68390f8ed2ad.slice/crio-dd64e6a87906aab3c2ce6af8f20d5c71ad4cd6eb144bd59c99fb4f65600f9b79 WatchSource:0}: Error finding container dd64e6a87906aab3c2ce6af8f20d5c71ad4cd6eb144bd59c99fb4f65600f9b79: Status 404 returned error can't find the container with id dd64e6a87906aab3c2ce6af8f20d5c71ad4cd6eb144bd59c99fb4f65600f9b79
Oct 30 12:59:16.938640 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:16.938617    4945 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 12:59:17.748092 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:17.748055    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r" event={"ID":"a39c4f07-8275-4aff-8cb5-68390f8ed2ad","Type":"ContainerStarted","Data":"a49c80c7e0152733f4030e833d33538b9c36458699c8aac243835bff519a7ce3"}
Oct 30 12:59:17.748522 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:17.748114    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:17.748522 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:17.748130    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r" event={"ID":"a39c4f07-8275-4aff-8cb5-68390f8ed2ad","Type":"ContainerStarted","Data":"dd64e6a87906aab3c2ce6af8f20d5c71ad4cd6eb144bd59c99fb4f65600f9b79"}
Oct 30 12:59:17.790465 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:17.790411    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r" podStartSLOduration=1.332910871 podStartE2EDuration="1.790396939s" totalImagesPullingTime="457.486068ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:59:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:59:17.786371556 +0000 UTC m=+1402.364320647" watchObservedRunningTime="2026-10-30 12:59:17.790396939 +0000 UTC m=+1402.368346065"
Oct 30 12:59:23.761805 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:23.761775    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 12:59:26.196717 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:59:26.196680    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 12:59:31.800582 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:31.800541    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"]
Oct 30 12:59:46.826050 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:46.826002    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" containerID="cri-o://cdabb2129f093186645c10e691e36e6888f306281e3dbe293b8baf44d4fb3a55" gracePeriod=45
Oct 30 12:59:46.836618 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:46.836568    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main" probeResult="failure" output="Get \"https://10.132.0.85:8000/ready\": EOF"
Oct 30 12:59:47.112243 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.112162    4945 generic.go:356] "Generic (PLEG): container finished" podID="505509b6-733d-4056-94eb-756ffc629032" containerID="cdabb2129f093186645c10e691e36e6888f306281e3dbe293b8baf44d4fb3a55" exitCode=0
Oct 30 12:59:47.112354 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.112229    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" event={"ID":"505509b6-733d-4056-94eb-756ffc629032","Type":"ContainerDied","Data":"cdabb2129f093186645c10e691e36e6888f306281e3dbe293b8baf44d4fb3a55"}
Oct 30 12:59:47.136167 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.136139    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:59:47.145468 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.145441    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-model-cache\") pod \"505509b6-733d-4056-94eb-756ffc629032\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") "
Oct 30 12:59:47.145639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.145553    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-dshm\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-dshm\") pod \"505509b6-733d-4056-94eb-756ffc629032\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") "
Oct 30 12:59:47.145639 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.145582    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/505509b6-733d-4056-94eb-756ffc629032-tls-certs\" (UniqueName: \"kubernetes.io/secret/505509b6-733d-4056-94eb-756ffc629032-tls-certs\") pod \"505509b6-733d-4056-94eb-756ffc629032\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") "
Oct 30 12:59:47.145770 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.145656    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-home\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-home\") pod \"505509b6-733d-4056-94eb-756ffc629032\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") "
Oct 30 12:59:47.145770 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.145740    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-model-cache" pod "505509b6-733d-4056-94eb-756ffc629032" (UID: "505509b6-733d-4056-94eb-756ffc629032"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:59:47.145882 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.145823    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-tmp-dir\") pod \"505509b6-733d-4056-94eb-756ffc629032\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") "
Oct 30 12:59:47.145975 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.145953    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/505509b6-733d-4056-94eb-756ffc629032-kube-api-access-shbbl\" (UniqueName: \"kubernetes.io/projected/505509b6-733d-4056-94eb-756ffc629032-kube-api-access-shbbl\") pod \"505509b6-733d-4056-94eb-756ffc629032\" (UID: \"505509b6-733d-4056-94eb-756ffc629032\") "
Oct 30 12:59:47.146108 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.146066    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-home" pod "505509b6-733d-4056-94eb-756ffc629032" (UID: "505509b6-733d-4056-94eb-756ffc629032"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:59:47.146468 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.146452    4945 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-model-cache\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:59:47.146528 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.146469    4945 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-home\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:59:47.146633 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.146610    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-tmp-dir" pod "505509b6-733d-4056-94eb-756ffc629032" (UID: "505509b6-733d-4056-94eb-756ffc629032"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:59:47.148139 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.148117    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-dshm" pod "505509b6-733d-4056-94eb-756ffc629032" (UID: "505509b6-733d-4056-94eb-756ffc629032"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 12:59:47.148367 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.148343    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/505509b6-733d-4056-94eb-756ffc629032-kube-api-access-shbbl" pod "505509b6-733d-4056-94eb-756ffc629032" (UID: "505509b6-733d-4056-94eb-756ffc629032"). InnerVolumeSpecName "kube-api-access-shbbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 12:59:47.150048 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.150024    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/505509b6-733d-4056-94eb-756ffc629032-tls-certs" pod "505509b6-733d-4056-94eb-756ffc629032" (UID: "505509b6-733d-4056-94eb-756ffc629032"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 12:59:47.247459 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.247427    4945 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-dshm\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:59:47.247459 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.247455    4945 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/505509b6-733d-4056-94eb-756ffc629032-tls-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:59:47.247459 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.247466    4945 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/505509b6-733d-4056-94eb-756ffc629032-tmp-dir\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:59:47.247717 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:47.247479    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shbbl\" (UniqueName: \"kubernetes.io/projected/505509b6-733d-4056-94eb-756ffc629032-kube-api-access-shbbl\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 12:59:48.136329 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:48.136283    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf" event={"ID":"505509b6-733d-4056-94eb-756ffc629032","Type":"ContainerDied","Data":"f9db7eaabb815c511d192c5a7430ed4d917368fa3f5755ad9a34f1483bd08656"}
Oct 30 12:59:48.136329 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:48.136354    4945 scope.go:122] "RemoveContainer" containerID="cdabb2129f093186645c10e691e36e6888f306281e3dbe293b8baf44d4fb3a55"
Oct 30 12:59:48.137147 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:48.136541    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"
Oct 30 12:59:48.190438 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:48.190394    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"]
Oct 30 12:59:48.201851 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:48.201813    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-7a04e002/e2e-lifecycle-model-7a04e002-kserve-5f9d94d9d9-n5tzf"]
Oct 30 12:59:49.550127 ip-10-0-142-36 kubenswrapper[4945]: I1030 12:59:49.550096    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="505509b6-733d-4056-94eb-756ffc629032" path="/var/lib/kubelet/pods/505509b6-733d-4056-94eb-756ffc629032/volumes"
Oct 30 12:59:56.196975 ip-10-0-142-36 kubenswrapper[4945]: E1030 12:59:56.196937    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:00:00.175093 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:00.175052    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"]
Oct 30 13:00:00.175947 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:00.175923    4945 cpu_manager.go:414] "containerMap: removing container" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main"
Oct 30 13:00:00.175947 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:00.175944    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main"
Oct 30 13:00:00.176243 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:00.176214    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="505509b6-733d-4056-94eb-756ffc629032" containerName="main"
Oct 30 13:00:00.182164 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:00.182140    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"
Oct 30 13:00:00.209528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:00.209493    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"]
Oct 30 13:00:00.251216 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:00.251188    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pfwpx\""
Oct 30 13:00:00.302329 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:00.302295    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pk79\" (UniqueName: \"kubernetes.io/projected/6b8ed252-f4f4-43d3-8f30-93aabe4a30b4-kube-api-access-4pk79\") pod \"maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c\" (UID: \"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"
Oct 30 13:00:00.302625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:00.302583    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ed252-f4f4-43d3-8f30-93aabe4a30b4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c\" (UID: \"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"
Oct 30 13:00:00.404049 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:00.404013    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ed252-f4f4-43d3-8f30-93aabe4a30b4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c\" (UID: \"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"
Oct 30 13:00:00.404225 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:00.404116    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pk79\" (UniqueName: \"kubernetes.io/projected/6b8ed252-f4f4-43d3-8f30-93aabe4a30b4-kube-api-access-4pk79\") pod \"maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c\" (UID: \"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"
Oct 30 13:00:00.404652 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:00.404630    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ed252-f4f4-43d3-8f30-93aabe4a30b4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c\" (UID: \"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"
Oct 30 13:00:00.424361 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:00.424333    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pk79\" (UniqueName: \"kubernetes.io/projected/6b8ed252-f4f4-43d3-8f30-93aabe4a30b4-kube-api-access-4pk79\") pod \"maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c\" (UID: \"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"
Oct 30 13:00:00.508461 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:00.508432    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"
Oct 30 13:00:00.865379 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:00.865352    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"]
Oct 30 13:00:00.866752 ip-10-0-142-36 kubenswrapper[4945]: W1030 13:00:00.866724    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b8ed252_f4f4_43d3_8f30_93aabe4a30b4.slice/crio-9e8092955d2246b5c41a64cbcdf87fc501a16f20dabce72673c48eaa7cc365e6 WatchSource:0}: Error finding container 9e8092955d2246b5c41a64cbcdf87fc501a16f20dabce72673c48eaa7cc365e6: Status 404 returned error can't find the container with id 9e8092955d2246b5c41a64cbcdf87fc501a16f20dabce72673c48eaa7cc365e6
Oct 30 13:00:01.249430 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:01.249393    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c" event={"ID":"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4","Type":"ContainerStarted","Data":"9e8092955d2246b5c41a64cbcdf87fc501a16f20dabce72673c48eaa7cc365e6"}
Oct 30 13:00:03.799945 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:03.799910    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"]
Oct 30 13:00:03.806718 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:03.806692    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:03.823094 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:03.823069    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"]
Oct 30 13:00:03.835939 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:03.835909    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-026c2f-serving-cert\""
Oct 30 13:00:03.866200 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:03.866162    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpnjq\" (UniqueName: \"kubernetes.io/projected/579928aa-fcf1-4385-95ac-5d8f8648e5be-kube-api-access-wpnjq\") pod \"maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:03.866358 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:03.866251    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/579928aa-fcf1-4385-95ac-5d8f8648e5be-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:03.866523 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:03.866441    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-maas-api-tls\") pod \"maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:03.866642 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:03.866567    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-metrics-certs\") pod \"maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:03.968033 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:03.967955    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpnjq\" (UniqueName: \"kubernetes.io/projected/579928aa-fcf1-4385-95ac-5d8f8648e5be-kube-api-access-wpnjq\") pod \"maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:03.968033 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:03.968026    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/579928aa-fcf1-4385-95ac-5d8f8648e5be-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:03.968261 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:03.968150    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-maas-api-tls\") pod \"maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:03.968261 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:03.968178    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-metrics-certs\") pod \"maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:03.968261 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:00:03.968235    4945 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-serving-cert: secret "maas-api-e2e-ipp-a-026c2f-serving-cert" not found
Oct 30 13:00:03.968424 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:00:03.968299    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-maas-api-tls podName:579928aa-fcf1-4385-95ac-5d8f8648e5be nodeName:}" failed. No retries permitted until 2026-10-30 13:00:04.46828107 +0000 UTC m=+1449.046230157 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-maas-api-tls") pod "maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd" (UID: "579928aa-fcf1-4385-95ac-5d8f8648e5be") : secret "maas-api-e2e-ipp-a-026c2f-serving-cert" not found
Oct 30 13:00:03.968766 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:03.968744    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/579928aa-fcf1-4385-95ac-5d8f8648e5be-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:03.970688 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:03.970670    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-metrics-certs\") pod \"maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:03.995957 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:03.995932    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpnjq\" (UniqueName: \"kubernetes.io/projected/579928aa-fcf1-4385-95ac-5d8f8648e5be-kube-api-access-wpnjq\") pod \"maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:04.274048 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:04.273939    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c" event={"ID":"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4","Type":"ContainerStarted","Data":"28f98550c5abb483c451850a7b027f87fe601020d6a6129646da106761064ae4"}
Oct 30 13:00:04.317461 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:04.317408    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c" podStartSLOduration=1.479721156 podStartE2EDuration="4.317394605s" totalImagesPullingTime="2.837673449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:00:04.314392585 +0000 UTC m=+1448.892341675" watchObservedRunningTime="2026-10-30 13:00:04.317394605 +0000 UTC m=+1448.895343693"
Oct 30 13:00:04.483565 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:04.483532    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-maas-api-tls\") pod \"maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:04.486097 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:04.486068    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-maas-api-tls\") pod \"maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:04.724539 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:04.724290    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:05.081477 ip-10-0-142-36 kubenswrapper[4945]: W1030 13:00:05.081445    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579928aa_fcf1_4385_95ac_5d8f8648e5be.slice/crio-117d999e25479ba9ce6e4c298ecc59ccad36ff801e3ace6fa241f7ca11096733 WatchSource:0}: Error finding container 117d999e25479ba9ce6e4c298ecc59ccad36ff801e3ace6fa241f7ca11096733: Status 404 returned error can't find the container with id 117d999e25479ba9ce6e4c298ecc59ccad36ff801e3ace6fa241f7ca11096733
Oct 30 13:00:05.081828 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:05.081619    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"]
Oct 30 13:00:05.290047 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:05.290004    4945 generic.go:356] "Generic (PLEG): container finished" podID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4" containerID="28f98550c5abb483c451850a7b027f87fe601020d6a6129646da106761064ae4" exitCode=60
Oct 30 13:00:05.290242 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:05.290165    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c" event={"ID":"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4","Type":"ContainerDied","Data":"28f98550c5abb483c451850a7b027f87fe601020d6a6129646da106761064ae4"}
Oct 30 13:00:05.290854 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:05.290823    4945 scope.go:122] "RemoveContainer" containerID="28f98550c5abb483c451850a7b027f87fe601020d6a6129646da106761064ae4"
Oct 30 13:00:05.295904 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:05.295869    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd" event={"ID":"579928aa-fcf1-4385-95ac-5d8f8648e5be","Type":"ContainerStarted","Data":"117d999e25479ba9ce6e4c298ecc59ccad36ff801e3ace6fa241f7ca11096733"}
Oct 30 13:00:06.305457 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:06.305413    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c" event={"ID":"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4","Type":"ContainerStarted","Data":"77a69029e6fffb4c03bb668521a057606ef9d58cba1c303a7c4a05297b933103"}
Oct 30 13:00:06.307707 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:06.307677    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd" event={"ID":"579928aa-fcf1-4385-95ac-5d8f8648e5be","Type":"ContainerStarted","Data":"82c0014e965898e5f4c5428011689faa605c101bee21af178462e1773ffdbc05"}
Oct 30 13:00:06.307849 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:06.307828    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:06.380932 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:06.380883    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd" podStartSLOduration=3.025520606 podStartE2EDuration="3.38086705s" totalImagesPullingTime="355.346444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:00:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:00:06.379214553 +0000 UTC m=+1450.957163642" watchObservedRunningTime="2026-10-30 13:00:06.38086705 +0000 UTC m=+1450.958816139"
Oct 30 13:00:07.336897 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:07.322483    4945 generic.go:356] "Generic (PLEG): container finished" podID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4" containerID="77a69029e6fffb4c03bb668521a057606ef9d58cba1c303a7c4a05297b933103" exitCode=60
Oct 30 13:00:07.336897 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:07.322703    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c" event={"ID":"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4","Type":"ContainerDied","Data":"77a69029e6fffb4c03bb668521a057606ef9d58cba1c303a7c4a05297b933103"}
Oct 30 13:00:07.336897 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:07.322925    4945 scope.go:122] "RemoveContainer" containerID="28f98550c5abb483c451850a7b027f87fe601020d6a6129646da106761064ae4"
Oct 30 13:00:07.336897 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:07.325144    4945 scope.go:122] "RemoveContainer" containerID="77a69029e6fffb4c03bb668521a057606ef9d58cba1c303a7c4a05297b933103"
Oct 30 13:00:07.336897 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:00:07.325582    4945 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c_odh-ai-gateway-infra(6b8ed252-f4f4-43d3-8f30-93aabe4a30b4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c" podUID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4"
Oct 30 13:00:08.331588 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:08.331559    4945 scope.go:122] "RemoveContainer" containerID="77a69029e6fffb4c03bb668521a057606ef9d58cba1c303a7c4a05297b933103"
Oct 30 13:00:08.331953 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:00:08.331928    4945 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c_odh-ai-gateway-infra(6b8ed252-f4f4-43d3-8f30-93aabe4a30b4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c" podUID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4"
Oct 30 13:00:11.977626 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:11.975513    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:12.046629 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:12.044786    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:00:12.054748 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:12.054705    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:12.375870 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:12.375784    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:00:12.787110 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:12.787064    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:12.789674 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:12.789639    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:12.909243 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:12.909191    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:12.933436 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:12.933388    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:12.988737 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:12.988692    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:12.989787 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:12.989760    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 13:00:13.056453 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:13.056360    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:13.067918 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:13.065810    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 13:00:13.537125 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:13.537080    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:13.827668 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:13.801998    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:13.887620 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:13.806726    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 13:00:13.920622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:13.888062    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 13:00:13.920622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:13.834035    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 13:00:13.920622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:13.909180    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 13:00:13.937938 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:13.937890    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:13.974623 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:13.942494    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 13:00:14.020102 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:14.020063    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 13:00:14.021373 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:14.021341    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": context deadline exceeded"
Oct 30 13:00:14.063378 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:14.062766    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:00:14.567926 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:14.567885    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:14.578435 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:14.572826    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 13:00:14.902638 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:14.891707    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:14.921759 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:14.921706    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:14.956762 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:14.956712    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:15.028622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:15.021460    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:15.066853 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:15.066806    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:15.397778 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:15.389228    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"]
Oct 30 13:00:15.400501 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:15.400465    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x" podUID="8293650f-2035-41cb-8db9-7942342fd116" containerName="istio-proxy" containerID="cri-o://235031e96c91f1a041fa65a912f5951f66cd9f913004f7e14316b523f02c3cb3" gracePeriod=30
Oct 30 13:00:15.592760 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:15.592707    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 13:00:15.808771 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:15.808716    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:15.860076 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:15.860046    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 13:00:15.899061 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:15.899021    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 13:00:15.907977 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:15.907944    4945 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7d9897b54c-wtc5x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:00:15.908231 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:15.908201    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x" podUID="8293650f-2035-41cb-8db9-7942342fd116" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:00:15.941726 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:15.941669    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:15.963744 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:15.963693    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:16.035969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:16.035048    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:16.086816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:16.068625    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:16.362021 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:16.278385    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"]
Oct 30 13:00:16.396612 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:16.396546    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj" podUID="6ddab487-2426-4e9f-9c72-e8151f5619cb" containerName="istio-proxy" containerID="cri-o://2342e8021585a636f7f5e7541464fe17f6adfbce78c49102eb7bcd904d12df77" gracePeriod=30
Oct 30 13:00:16.825626 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:16.824693    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:16.831679 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:16.826580    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 13:00:16.846061 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:16.845680    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:00:16.846061 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:16.845747    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:16.846463 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:16.846372    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:00:16.846463 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:16.846416    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:16.890818 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:16.890691    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:00:16.890818 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:16.890751    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:16.900690 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:16.899692    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 13:00:16.952727 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:16.950203    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:17.093129 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:17.088300    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:17.180628 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:17.169691    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:00:17.224104 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:17.222707    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:00:17.224104 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:17.222776    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk" podUID="492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:17.290637 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:17.090575    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 13:00:17.437687 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:17.430208    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:00:17.437687 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:17.430284    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd" podUID="b107d489-5cd0-492b-8cab-01d603040f8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:17.608914 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:17.608848    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 13:00:17.646829 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:17.636644    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/health\": context deadline exceeded"
Oct 30 13:00:17.836624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:17.825122    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 13:00:17.915842 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:17.900339    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:17.959795 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:17.959740    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 13:00:18.091679 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:18.088681    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:18.171743 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:18.171692    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:00:18.257877 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:18.257829    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:18.335861 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:18.335823    4945 patch_prober.go:29] interesting pod/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:00:18.336151 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:18.336119    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9" podUID="237869eb-18ff-4f5a-87e8-e7d8c97bd088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:18.614624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:18.611593    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 13:00:18.614624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:18.612167    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 13:00:18.889691 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:18.887735    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:18.925878 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:18.925830    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:18.965490 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:18.965441    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 13:00:19.089732 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:19.089682    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:19.173796 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:19.173691    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:19.265013 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:19.264910    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:19.618625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:19.618083    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:19.813742 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:19.813690    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:19.902684 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:19.902576    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:19.938904 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:19.938860    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:19.970080 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:19.970035    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:20.101744 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.101697    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 13:00:20.174995 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.174799    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:20.208566 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.208533    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 13:00:20.317228 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.316691    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:00:20.323316 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.322129    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 13:00:20.340556 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.340530    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 13:00:20.428881 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.428272    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 13:00:20.538697 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.531500    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 13:00:20.538697 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.532942    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 13:00:20.538697 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.537374    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 13:00:20.543228 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.542995    4945 scope.go:122] "RemoveContainer" containerID="77a69029e6fffb4c03bb668521a057606ef9d58cba1c303a7c4a05297b933103"
Oct 30 13:00:20.873933 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.873904    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 13:00:20.939412 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.939376    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-istio-envoy\") pod \"8293650f-2035-41cb-8db9-7942342fd116\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") "
Oct 30 13:00:20.939620 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.939423    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-istio-data\") pod \"8293650f-2035-41cb-8db9-7942342fd116\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") "
Oct 30 13:00:20.939620 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.939459    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-credential-socket\") pod \"8293650f-2035-41cb-8db9-7942342fd116\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") "
Oct 30 13:00:20.939620 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.939510    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8293650f-2035-41cb-8db9-7942342fd116-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8293650f-2035-41cb-8db9-7942342fd116-istio-podinfo\") pod \"8293650f-2035-41cb-8db9-7942342fd116\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") "
Oct 30 13:00:20.939620 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.939540    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-workload-certs\") pod \"8293650f-2035-41cb-8db9-7942342fd116\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") "
Oct 30 13:00:20.939620 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.939568    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8293650f-2035-41cb-8db9-7942342fd116-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8293650f-2035-41cb-8db9-7942342fd116-istiod-ca-cert\") pod \"8293650f-2035-41cb-8db9-7942342fd116\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") "
Oct 30 13:00:20.939919 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.939683    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8293650f-2035-41cb-8db9-7942342fd116-istio-token\" (UniqueName: \"kubernetes.io/projected/8293650f-2035-41cb-8db9-7942342fd116-istio-token\") pod \"8293650f-2035-41cb-8db9-7942342fd116\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") "
Oct 30 13:00:20.939919 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.939737    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-workload-socket\") pod \"8293650f-2035-41cb-8db9-7942342fd116\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") "
Oct 30 13:00:20.939919 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.939795    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8293650f-2035-41cb-8db9-7942342fd116-kube-api-access-lw6kh\" (UniqueName: \"kubernetes.io/projected/8293650f-2035-41cb-8db9-7942342fd116-kube-api-access-lw6kh\") pod \"8293650f-2035-41cb-8db9-7942342fd116\" (UID: \"8293650f-2035-41cb-8db9-7942342fd116\") "
Oct 30 13:00:20.940157 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.940124    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-workload-certs" pod "8293650f-2035-41cb-8db9-7942342fd116" (UID: "8293650f-2035-41cb-8db9-7942342fd116"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:00:20.940157 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.940142    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8293650f-2035-41cb-8db9-7942342fd116-istiod-ca-cert" pod "8293650f-2035-41cb-8db9-7942342fd116" (UID: "8293650f-2035-41cb-8db9-7942342fd116"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:00:20.940952 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.940354    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-workload-socket" pod "8293650f-2035-41cb-8db9-7942342fd116" (UID: "8293650f-2035-41cb-8db9-7942342fd116"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:00:20.940952 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.940835    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:20.940952 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.940857    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8293650f-2035-41cb-8db9-7942342fd116-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:20.940952 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.940871    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:20.940952 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.940925    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-credential-socket" pod "8293650f-2035-41cb-8db9-7942342fd116" (UID: "8293650f-2035-41cb-8db9-7942342fd116"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:00:20.941551 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.941526    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-istio-data" pod "8293650f-2035-41cb-8db9-7942342fd116" (UID: "8293650f-2035-41cb-8db9-7942342fd116"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:00:20.946919 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.942692    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-istio-envoy" pod "8293650f-2035-41cb-8db9-7942342fd116" (UID: "8293650f-2035-41cb-8db9-7942342fd116"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:00:20.946919 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.942719    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8293650f-2035-41cb-8db9-7942342fd116-istio-token" pod "8293650f-2035-41cb-8db9-7942342fd116" (UID: "8293650f-2035-41cb-8db9-7942342fd116"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:00:20.946919 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.942849    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8293650f-2035-41cb-8db9-7942342fd116-kube-api-access-lw6kh" pod "8293650f-2035-41cb-8db9-7942342fd116" (UID: "8293650f-2035-41cb-8db9-7942342fd116"). InnerVolumeSpecName "kube-api-access-lw6kh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:00:20.946919 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:20.942940    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8293650f-2035-41cb-8db9-7942342fd116-istio-podinfo" pod "8293650f-2035-41cb-8db9-7942342fd116" (UID: "8293650f-2035-41cb-8db9-7942342fd116"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:00:21.043199 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.043109    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:21.043199 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.043154    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:21.043199 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.043170    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8293650f-2035-41cb-8db9-7942342fd116-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:21.043199 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.043186    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8293650f-2035-41cb-8db9-7942342fd116-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:21.043199 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.043201    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8293650f-2035-41cb-8db9-7942342fd116-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:21.043493 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.043215    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lw6kh\" (UniqueName: \"kubernetes.io/projected/8293650f-2035-41cb-8db9-7942342fd116-kube-api-access-lw6kh\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:21.066925 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.066885    4945 generic.go:356] "Generic (PLEG): container finished" podID="8293650f-2035-41cb-8db9-7942342fd116" containerID="235031e96c91f1a041fa65a912f5951f66cd9f913004f7e14316b523f02c3cb3" exitCode=0
Oct 30 13:00:21.067107 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.066946    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x" event={"ID":"8293650f-2035-41cb-8db9-7942342fd116","Type":"ContainerDied","Data":"235031e96c91f1a041fa65a912f5951f66cd9f913004f7e14316b523f02c3cb3"}
Oct 30 13:00:21.067107 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.067051    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x" event={"ID":"8293650f-2035-41cb-8db9-7942342fd116","Type":"ContainerDied","Data":"9b2d61bd088e792b2cb6292d26aa7971cec7e95f52c98b0d2da0966531993808"}
Oct 30 13:00:21.067107 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.067079    4945 scope.go:122] "RemoveContainer" containerID="235031e96c91f1a041fa65a912f5951f66cd9f913004f7e14316b523f02c3cb3"
Oct 30 13:00:21.067277 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.067234    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"
Oct 30 13:00:21.086885 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.085674    4945 scope.go:122] "RemoveContainer" containerID="bd6c28a719533806985d76d39806e65acc3e2f42292f2897100320590bfeb490"
Oct 30 13:00:21.106228 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.106202    4945 scope.go:122] "RemoveContainer" containerID="235031e96c91f1a041fa65a912f5951f66cd9f913004f7e14316b523f02c3cb3"
Oct 30 13:00:21.106783 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:00:21.106752    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"235031e96c91f1a041fa65a912f5951f66cd9f913004f7e14316b523f02c3cb3\": container with ID starting with 235031e96c91f1a041fa65a912f5951f66cd9f913004f7e14316b523f02c3cb3 not found: ID does not exist" containerID="235031e96c91f1a041fa65a912f5951f66cd9f913004f7e14316b523f02c3cb3"
Oct 30 13:00:21.106890 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.106818    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"235031e96c91f1a041fa65a912f5951f66cd9f913004f7e14316b523f02c3cb3"} err="failed to get container status \"235031e96c91f1a041fa65a912f5951f66cd9f913004f7e14316b523f02c3cb3\": rpc error: code = NotFound desc = could not find container \"235031e96c91f1a041fa65a912f5951f66cd9f913004f7e14316b523f02c3cb3\": container with ID starting with 235031e96c91f1a041fa65a912f5951f66cd9f913004f7e14316b523f02c3cb3 not found: ID does not exist"
Oct 30 13:00:21.106890 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.106845    4945 scope.go:122] "RemoveContainer" containerID="bd6c28a719533806985d76d39806e65acc3e2f42292f2897100320590bfeb490"
Oct 30 13:00:21.107352 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:00:21.107248    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd6c28a719533806985d76d39806e65acc3e2f42292f2897100320590bfeb490\": container with ID starting with bd6c28a719533806985d76d39806e65acc3e2f42292f2897100320590bfeb490 not found: ID does not exist" containerID="bd6c28a719533806985d76d39806e65acc3e2f42292f2897100320590bfeb490"
Oct 30 13:00:21.107352 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.107301    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd6c28a719533806985d76d39806e65acc3e2f42292f2897100320590bfeb490"} err="failed to get container status \"bd6c28a719533806985d76d39806e65acc3e2f42292f2897100320590bfeb490\": rpc error: code = NotFound desc = could not find container \"bd6c28a719533806985d76d39806e65acc3e2f42292f2897100320590bfeb490\": container with ID starting with bd6c28a719533806985d76d39806e65acc3e2f42292f2897100320590bfeb490 not found: ID does not exist"
Oct 30 13:00:21.125482 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.125453    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"]
Oct 30 13:00:21.129535 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.129513    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wtc5x"]
Oct 30 13:00:21.552499 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.552465    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8293650f-2035-41cb-8db9-7942342fd116" path="/var/lib/kubelet/pods/8293650f-2035-41cb-8db9-7942342fd116/volumes"
Oct 30 13:00:21.822262 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.822192    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 13:00:21.856180 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.856149    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-credential-socket\") pod \"6ddab487-2426-4e9f-9c72-e8151f5619cb\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") "
Oct 30 13:00:21.856317 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.856214    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-envoy\") pod \"6ddab487-2426-4e9f-9c72-e8151f5619cb\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") "
Oct 30 13:00:21.856317 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.856252    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-data\") pod \"6ddab487-2426-4e9f-9c72-e8151f5619cb\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") "
Oct 30 13:00:21.856450 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.856339    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-token\" (UniqueName: \"kubernetes.io/projected/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-token\") pod \"6ddab487-2426-4e9f-9c72-e8151f5619cb\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") "
Oct 30 13:00:21.856450 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.856423    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ddab487-2426-4e9f-9c72-e8151f5619cb-kube-api-access-qhwrf\" (UniqueName: \"kubernetes.io/projected/6ddab487-2426-4e9f-9c72-e8151f5619cb-kube-api-access-qhwrf\") pod \"6ddab487-2426-4e9f-9c72-e8151f5619cb\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") "
Oct 30 13:00:21.856556 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.856468    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ddab487-2426-4e9f-9c72-e8151f5619cb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ddab487-2426-4e9f-9c72-e8151f5619cb-istiod-ca-cert\") pod \"6ddab487-2426-4e9f-9c72-e8151f5619cb\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") "
Oct 30 13:00:21.856556 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.856475    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-credential-socket" pod "6ddab487-2426-4e9f-9c72-e8151f5619cb" (UID: "6ddab487-2426-4e9f-9c72-e8151f5619cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:00:21.856714 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.856683    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-workload-socket\") pod \"6ddab487-2426-4e9f-9c72-e8151f5619cb\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") "
Oct 30 13:00:21.856759 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.856695    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-data" pod "6ddab487-2426-4e9f-9c72-e8151f5619cb" (UID: "6ddab487-2426-4e9f-9c72-e8151f5619cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:00:21.856759 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.856726    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-workload-certs\") pod \"6ddab487-2426-4e9f-9c72-e8151f5619cb\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") "
Oct 30 13:00:21.856859 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.856831    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-podinfo\") pod \"6ddab487-2426-4e9f-9c72-e8151f5619cb\" (UID: \"6ddab487-2426-4e9f-9c72-e8151f5619cb\") "
Oct 30 13:00:21.859643 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.856948    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ddab487-2426-4e9f-9c72-e8151f5619cb-istiod-ca-cert" pod "6ddab487-2426-4e9f-9c72-e8151f5619cb" (UID: "6ddab487-2426-4e9f-9c72-e8151f5619cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:00:21.859643 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.857055    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-workload-socket" pod "6ddab487-2426-4e9f-9c72-e8151f5619cb" (UID: "6ddab487-2426-4e9f-9c72-e8151f5619cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:00:21.859643 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.857132    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-workload-certs" pod "6ddab487-2426-4e9f-9c72-e8151f5619cb" (UID: "6ddab487-2426-4e9f-9c72-e8151f5619cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:00:21.859643 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.857528    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ddab487-2426-4e9f-9c72-e8151f5619cb-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:21.859643 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.857543    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:21.859643 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.857556    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:21.859643 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.857566    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:21.859643 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.857578    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:21.859643 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.859188    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-token" pod "6ddab487-2426-4e9f-9c72-e8151f5619cb" (UID: "6ddab487-2426-4e9f-9c72-e8151f5619cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:00:21.859643 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.859351    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ddab487-2426-4e9f-9c72-e8151f5619cb-kube-api-access-qhwrf" pod "6ddab487-2426-4e9f-9c72-e8151f5619cb" (UID: "6ddab487-2426-4e9f-9c72-e8151f5619cb"). InnerVolumeSpecName "kube-api-access-qhwrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:00:21.859643 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.859398    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-podinfo" pod "6ddab487-2426-4e9f-9c72-e8151f5619cb" (UID: "6ddab487-2426-4e9f-9c72-e8151f5619cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:00:21.860293 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.859730    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-envoy" pod "6ddab487-2426-4e9f-9c72-e8151f5619cb" (UID: "6ddab487-2426-4e9f-9c72-e8151f5619cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:00:21.958441 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.958414    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:21.958441 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.958440    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhwrf\" (UniqueName: \"kubernetes.io/projected/6ddab487-2426-4e9f-9c72-e8151f5619cb-kube-api-access-qhwrf\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:21.958657 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.958451    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:21.958657 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:21.958461    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ddab487-2426-4e9f-9c72-e8151f5619cb-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:22.077440 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:22.077359    4945 generic.go:356] "Generic (PLEG): container finished" podID="6ddab487-2426-4e9f-9c72-e8151f5619cb" containerID="2342e8021585a636f7f5e7541464fe17f6adfbce78c49102eb7bcd904d12df77" exitCode=0
Oct 30 13:00:22.077440 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:22.077412    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"
Oct 30 13:00:22.077655 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:22.077442    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj" event={"ID":"6ddab487-2426-4e9f-9c72-e8151f5619cb","Type":"ContainerDied","Data":"2342e8021585a636f7f5e7541464fe17f6adfbce78c49102eb7bcd904d12df77"}
Oct 30 13:00:22.077655 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:22.077494    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj" event={"ID":"6ddab487-2426-4e9f-9c72-e8151f5619cb","Type":"ContainerDied","Data":"78f29e2e836420f3f23cc9f6efc74361fdc050e0a662f5e00cd30d01b6b51c2b"}
Oct 30 13:00:22.077655 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:22.077525    4945 scope.go:122] "RemoveContainer" containerID="2342e8021585a636f7f5e7541464fe17f6adfbce78c49102eb7bcd904d12df77"
Oct 30 13:00:22.084330 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:22.083956    4945 generic.go:356] "Generic (PLEG): container finished" podID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4" containerID="87301e41f8a916b484373986992bd1bc59aee2e56e0214c8a26682f0957de519" exitCode=60
Oct 30 13:00:22.084330 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:22.083990    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c" event={"ID":"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4","Type":"ContainerDied","Data":"87301e41f8a916b484373986992bd1bc59aee2e56e0214c8a26682f0957de519"}
Oct 30 13:00:22.084501 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:22.084457    4945 scope.go:122] "RemoveContainer" containerID="87301e41f8a916b484373986992bd1bc59aee2e56e0214c8a26682f0957de519"
Oct 30 13:00:22.084818 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:00:22.084790    4945 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c_odh-ai-gateway-infra(6b8ed252-f4f4-43d3-8f30-93aabe4a30b4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c" podUID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4"
Oct 30 13:00:22.093802 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:22.093767    4945 scope.go:122] "RemoveContainer" containerID="2342e8021585a636f7f5e7541464fe17f6adfbce78c49102eb7bcd904d12df77"
Oct 30 13:00:22.094293 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:00:22.094239    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2342e8021585a636f7f5e7541464fe17f6adfbce78c49102eb7bcd904d12df77\": container with ID starting with 2342e8021585a636f7f5e7541464fe17f6adfbce78c49102eb7bcd904d12df77 not found: ID does not exist" containerID="2342e8021585a636f7f5e7541464fe17f6adfbce78c49102eb7bcd904d12df77"
Oct 30 13:00:22.094293 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:22.094272    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2342e8021585a636f7f5e7541464fe17f6adfbce78c49102eb7bcd904d12df77"} err="failed to get container status \"2342e8021585a636f7f5e7541464fe17f6adfbce78c49102eb7bcd904d12df77\": rpc error: code = NotFound desc = could not find container \"2342e8021585a636f7f5e7541464fe17f6adfbce78c49102eb7bcd904d12df77\": container with ID starting with 2342e8021585a636f7f5e7541464fe17f6adfbce78c49102eb7bcd904d12df77 not found: ID does not exist"
Oct 30 13:00:22.094293 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:22.094289    4945 scope.go:122] "RemoveContainer" containerID="77a69029e6fffb4c03bb668521a057606ef9d58cba1c303a7c4a05297b933103"
Oct 30 13:00:22.166139 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:22.166108    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"]
Oct 30 13:00:22.169873 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:22.169851    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7a04e002-openshift-default-6669855c89-988sj"]
Oct 30 13:00:23.105343 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.105310    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"]
Oct 30 13:00:23.106294 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.106269    4945 cpu_manager.go:414] "containerMap: removing container" podUID="6ddab487-2426-4e9f-9c72-e8151f5619cb" containerName="istio-proxy"
Oct 30 13:00:23.106294 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.106293    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ddab487-2426-4e9f-9c72-e8151f5619cb" containerName="istio-proxy"
Oct 30 13:00:23.106527 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.106321    4945 cpu_manager.go:414] "containerMap: removing container" podUID="8293650f-2035-41cb-8db9-7942342fd116" containerName="istio-proxy"
Oct 30 13:00:23.106527 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.106329    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8293650f-2035-41cb-8db9-7942342fd116" containerName="istio-proxy"
Oct 30 13:00:23.106527 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.106522    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="8293650f-2035-41cb-8db9-7942342fd116" containerName="istio-proxy"
Oct 30 13:00:23.106759 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.106538    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ddab487-2426-4e9f-9c72-e8151f5619cb" containerName="istio-proxy"
Oct 30 13:00:23.106759 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.106551    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="8293650f-2035-41cb-8db9-7942342fd116" containerName="istio-proxy"
Oct 30 13:00:23.125576 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.114313    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:23.138950 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.138910    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"]
Oct 30 13:00:23.139473 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.139437    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-096cc3-serving-cert\""
Oct 30 13:00:23.152466 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.152440    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"]
Oct 30 13:00:23.198651 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.195178    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-maas-api-tls\") pod \"maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:23.198651 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.195273    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-metrics-certs\") pod \"maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:23.198651 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.195384    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2000e193-dc40-4807-867f-48bad0847b73-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:23.198651 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.195435    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52lw7\" (UniqueName: \"kubernetes.io/projected/2000e193-dc40-4807-867f-48bad0847b73-kube-api-access-52lw7\") pod \"maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:23.306657 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.299060    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-maas-api-tls\") pod \"maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:23.306657 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.299147    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-metrics-certs\") pod \"maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:23.306657 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.299256    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2000e193-dc40-4807-867f-48bad0847b73-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:23.306657 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.299303    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52lw7\" (UniqueName: \"kubernetes.io/projected/2000e193-dc40-4807-867f-48bad0847b73-kube-api-access-52lw7\") pod \"maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:23.306657 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:00:23.299780    4945 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-serving-cert: secret "maas-api-e2e-ipp-b-096cc3-serving-cert" not found
Oct 30 13:00:23.306657 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:00:23.299854    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-maas-api-tls podName:2000e193-dc40-4807-867f-48bad0847b73 nodeName:}" failed. No retries permitted until 2026-10-30 13:00:23.79983293 +0000 UTC m=+1468.377782014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-maas-api-tls") pod "maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7" (UID: "2000e193-dc40-4807-867f-48bad0847b73") : secret "maas-api-e2e-ipp-b-096cc3-serving-cert" not found
Oct 30 13:00:23.326960 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.326922    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2000e193-dc40-4807-867f-48bad0847b73-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:23.330313 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.330217    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-metrics-certs\") pod \"maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:23.333852 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.333788    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52lw7\" (UniqueName: \"kubernetes.io/projected/2000e193-dc40-4807-867f-48bad0847b73-kube-api-access-52lw7\") pod \"maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:23.380811 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.380785    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"
Oct 30 13:00:23.505619 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.504192    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b8ed252-f4f4-43d3-8f30-93aabe4a30b4-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ed252-f4f4-43d3-8f30-93aabe4a30b4-maas-api-service-ca\") pod \"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4\" (UID: \"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4\") "
Oct 30 13:00:23.505619 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.504311    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b8ed252-f4f4-43d3-8f30-93aabe4a30b4-kube-api-access-4pk79\" (UniqueName: \"kubernetes.io/projected/6b8ed252-f4f4-43d3-8f30-93aabe4a30b4-kube-api-access-4pk79\") pod \"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4\" (UID: \"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4\") "
Oct 30 13:00:23.535867 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.535819    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b8ed252-f4f4-43d3-8f30-93aabe4a30b4-maas-api-service-ca" pod "6b8ed252-f4f4-43d3-8f30-93aabe4a30b4" (UID: "6b8ed252-f4f4-43d3-8f30-93aabe4a30b4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:00:23.604678 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.601786    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b8ed252-f4f4-43d3-8f30-93aabe4a30b4-kube-api-access-4pk79" pod "6b8ed252-f4f4-43d3-8f30-93aabe4a30b4" (UID: "6b8ed252-f4f4-43d3-8f30-93aabe4a30b4"). InnerVolumeSpecName "kube-api-access-4pk79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:00:23.626531 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.626444    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ddab487-2426-4e9f-9c72-e8151f5619cb" path="/var/lib/kubelet/pods/6ddab487-2426-4e9f-9c72-e8151f5619cb/volumes"
Oct 30 13:00:23.644663 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.641247    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8ed252-f4f4-43d3-8f30-93aabe4a30b4-maas-api-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:23.644663 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.641284    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pk79\" (UniqueName: \"kubernetes.io/projected/6b8ed252-f4f4-43d3-8f30-93aabe4a30b4-kube-api-access-4pk79\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:23.845440 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.845402    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-maas-api-tls\") pod \"maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:23.847874 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:23.847849    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-maas-api-tls\") pod \"maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:24.045392 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:24.045355    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:24.120623 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:24.119933    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c" event={"ID":"6b8ed252-f4f4-43d3-8f30-93aabe4a30b4","Type":"ContainerDied","Data":"9e8092955d2246b5c41a64cbcdf87fc501a16f20dabce72673c48eaa7cc365e6"}
Oct 30 13:00:24.120623 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:24.120012    4945 scope.go:122] "RemoveContainer" containerID="87301e41f8a916b484373986992bd1bc59aee2e56e0214c8a26682f0957de519"
Oct 30 13:00:24.120623 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:24.120158    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"
Oct 30 13:00:24.248076 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:24.248017    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"]
Oct 30 13:00:24.253550 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:24.253457    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-f1222e-29889420-x6c2c"]
Oct 30 13:00:24.400049 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:24.399955    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"]
Oct 30 13:00:24.404919 ip-10-0-142-36 kubenswrapper[4945]: W1030 13:00:24.404129    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2000e193_dc40_4807_867f_48bad0847b73.slice/crio-399e198c3263632422c17e681f65918998311bf23b726cb2feed527f2e3cb752 WatchSource:0}: Error finding container 399e198c3263632422c17e681f65918998311bf23b726cb2feed527f2e3cb752: Status 404 returned error can't find the container with id 399e198c3263632422c17e681f65918998311bf23b726cb2feed527f2e3cb752
Oct 30 13:00:25.129708 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:25.129628    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7" event={"ID":"2000e193-dc40-4807-867f-48bad0847b73","Type":"ContainerStarted","Data":"3255a26944a66b5589c7a7e39c88099b6526d496ac562f2d77766d2430a3b16a"}
Oct 30 13:00:25.129708 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:25.129690    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7" event={"ID":"2000e193-dc40-4807-867f-48bad0847b73","Type":"ContainerStarted","Data":"399e198c3263632422c17e681f65918998311bf23b726cb2feed527f2e3cb752"}
Oct 30 13:00:25.130131 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:25.129713    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:25.175875 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:25.175438    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7" podStartSLOduration=1.802976211 podStartE2EDuration="2.175419953s" totalImagesPullingTime="372.443742ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:00:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:00:25.173346506 +0000 UTC m=+1469.751295601" watchObservedRunningTime="2026-10-30 13:00:25.175419953 +0000 UTC m=+1469.753369041"
Oct 30 13:00:25.555826 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:25.555786    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4" path="/var/lib/kubelet/pods/6b8ed252-f4f4-43d3-8f30-93aabe4a30b4/volumes"
Oct 30 13:00:26.195514 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:00:26.195449    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:00:31.142717 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:31.142684    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:00:33.306117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:33.306082    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"]
Oct 30 13:00:33.306575 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:33.306280    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8" podUID="e9cf7ee9-33f1-4291-834b-8289c4e7b4e0" containerName="maas-api" containerID="cri-o://3d8d9eaf6908b0980ae5760e4a9674fec60928f4d9e8a526ec59b09fde6ecc17" gracePeriod=30
Oct 30 13:00:33.675594 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:33.675565    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 13:00:33.780042 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:33.778133    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-maas-api-tls\") pod \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") "
Oct 30 13:00:33.780401 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:33.780380    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-metrics-certs\") pod \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") "
Oct 30 13:00:33.781022 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:33.780998    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-kube-api-access-d6hvp\" (UniqueName: \"kubernetes.io/projected/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-kube-api-access-d6hvp\") pod \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") "
Oct 30 13:00:33.782138 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:33.782021    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-maas-api-tls" pod "e9cf7ee9-33f1-4291-834b-8289c4e7b4e0" (UID: "e9cf7ee9-33f1-4291-834b-8289c4e7b4e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:00:33.782138 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:33.782097    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-openshift-service-ca" pod "e9cf7ee9-33f1-4291-834b-8289c4e7b4e0" (UID: "e9cf7ee9-33f1-4291-834b-8289c4e7b4e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:00:33.782336 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:33.782317    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-openshift-service-ca\") pod \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\" (UID: \"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0\") "
Oct 30 13:00:33.783257 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:33.783236    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:33.783397 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:33.783382    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:33.783656 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:33.783562    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-kube-api-access-d6hvp" pod "e9cf7ee9-33f1-4291-834b-8289c4e7b4e0" (UID: "e9cf7ee9-33f1-4291-834b-8289c4e7b4e0"). InnerVolumeSpecName "kube-api-access-d6hvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:00:33.785890 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:33.785852    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-metrics-certs" pod "e9cf7ee9-33f1-4291-834b-8289c4e7b4e0" (UID: "e9cf7ee9-33f1-4291-834b-8289c4e7b4e0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:00:33.891157 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:33.891065    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:33.891393 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:33.891374    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6hvp\" (UniqueName: \"kubernetes.io/projected/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0-kube-api-access-d6hvp\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:34.241900 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:34.241861    4945 generic.go:356] "Generic (PLEG): container finished" podID="e9cf7ee9-33f1-4291-834b-8289c4e7b4e0" containerID="3d8d9eaf6908b0980ae5760e4a9674fec60928f4d9e8a526ec59b09fde6ecc17" exitCode=0
Oct 30 13:00:34.242110 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:34.241944    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"
Oct 30 13:00:34.242110 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:34.241987    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8" event={"ID":"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0","Type":"ContainerDied","Data":"3d8d9eaf6908b0980ae5760e4a9674fec60928f4d9e8a526ec59b09fde6ecc17"}
Oct 30 13:00:34.242236 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:34.242129    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8" event={"ID":"e9cf7ee9-33f1-4291-834b-8289c4e7b4e0","Type":"ContainerDied","Data":"a43aae4d62e7b9b4404595387f872a798441c0278baa39900b528ba25d426a07"}
Oct 30 13:00:34.242236 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:34.242166    4945 scope.go:122] "RemoveContainer" containerID="3d8d9eaf6908b0980ae5760e4a9674fec60928f4d9e8a526ec59b09fde6ecc17"
Oct 30 13:00:34.255085 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:34.255064    4945 scope.go:122] "RemoveContainer" containerID="3d8d9eaf6908b0980ae5760e4a9674fec60928f4d9e8a526ec59b09fde6ecc17"
Oct 30 13:00:34.255413 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:00:34.255386    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d8d9eaf6908b0980ae5760e4a9674fec60928f4d9e8a526ec59b09fde6ecc17\": container with ID starting with 3d8d9eaf6908b0980ae5760e4a9674fec60928f4d9e8a526ec59b09fde6ecc17 not found: ID does not exist" containerID="3d8d9eaf6908b0980ae5760e4a9674fec60928f4d9e8a526ec59b09fde6ecc17"
Oct 30 13:00:34.255526 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:34.255421    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d8d9eaf6908b0980ae5760e4a9674fec60928f4d9e8a526ec59b09fde6ecc17"} err="failed to get container status \"3d8d9eaf6908b0980ae5760e4a9674fec60928f4d9e8a526ec59b09fde6ecc17\": rpc error: code = NotFound desc = could not find container \"3d8d9eaf6908b0980ae5760e4a9674fec60928f4d9e8a526ec59b09fde6ecc17\": container with ID starting with 3d8d9eaf6908b0980ae5760e4a9674fec60928f4d9e8a526ec59b09fde6ecc17 not found: ID does not exist"
Oct 30 13:00:34.305559 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:34.303891    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"]
Oct 30 13:00:34.310975 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:34.310945    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2bfb1b-7599dfb644-7hkw8"]
Oct 30 13:00:35.552714 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:35.552684    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9cf7ee9-33f1-4291-834b-8289c4e7b4e0" path="/var/lib/kubelet/pods/e9cf7ee9-33f1-4291-834b-8289c4e7b4e0/volumes"
Oct 30 13:00:40.775193 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:40.773540    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"]
Oct 30 13:00:40.775193 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:40.773824    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m" podUID="2bef9404-f32d-4093-a4d2-f63a966df637" containerName="istio-proxy" containerID="cri-o://5145345ea68150b357904a5f2f1b906ce1bd99c0e2c8a316f5900be0bd0662b2" gracePeriod=30
Oct 30 13:00:46.145553 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.145512    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 13:00:46.211592 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.211559    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2bef9404-f32d-4093-a4d2-f63a966df637-istio-token\" (UniqueName: \"kubernetes.io/projected/2bef9404-f32d-4093-a4d2-f63a966df637-istio-token\") pod \"2bef9404-f32d-4093-a4d2-f63a966df637\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") "
Oct 30 13:00:46.211790 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.211696    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-workload-certs\") pod \"2bef9404-f32d-4093-a4d2-f63a966df637\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") "
Oct 30 13:00:46.211790 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.211729    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2bef9404-f32d-4093-a4d2-f63a966df637-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bef9404-f32d-4093-a4d2-f63a966df637-istio-podinfo\") pod \"2bef9404-f32d-4093-a4d2-f63a966df637\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") "
Oct 30 13:00:46.211790 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.211780    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-credential-socket\") pod \"2bef9404-f32d-4093-a4d2-f63a966df637\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") "
Oct 30 13:00:46.211965 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.211902    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-istio-data\") pod \"2bef9404-f32d-4093-a4d2-f63a966df637\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") "
Oct 30 13:00:46.212018 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.211979    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-istio-envoy\") pod \"2bef9404-f32d-4093-a4d2-f63a966df637\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") "
Oct 30 13:00:46.212073 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.212015    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2bef9404-f32d-4093-a4d2-f63a966df637-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bef9404-f32d-4093-a4d2-f63a966df637-istiod-ca-cert\") pod \"2bef9404-f32d-4093-a4d2-f63a966df637\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") "
Oct 30 13:00:46.212157 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.212127    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2bef9404-f32d-4093-a4d2-f63a966df637-kube-api-access-b5756\" (UniqueName: \"kubernetes.io/projected/2bef9404-f32d-4093-a4d2-f63a966df637-kube-api-access-b5756\") pod \"2bef9404-f32d-4093-a4d2-f63a966df637\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") "
Oct 30 13:00:46.212217 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.212163    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-workload-socket\") pod \"2bef9404-f32d-4093-a4d2-f63a966df637\" (UID: \"2bef9404-f32d-4093-a4d2-f63a966df637\") "
Oct 30 13:00:46.212929 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.212902    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-workload-socket" pod "2bef9404-f32d-4093-a4d2-f63a966df637" (UID: "2bef9404-f32d-4093-a4d2-f63a966df637"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:00:46.213082 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.212916    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-credential-socket" pod "2bef9404-f32d-4093-a4d2-f63a966df637" (UID: "2bef9404-f32d-4093-a4d2-f63a966df637"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:00:46.213198 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.213180    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-istio-data" pod "2bef9404-f32d-4093-a4d2-f63a966df637" (UID: "2bef9404-f32d-4093-a4d2-f63a966df637"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:00:46.213281 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.213254    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bef9404-f32d-4093-a4d2-f63a966df637-istiod-ca-cert" pod "2bef9404-f32d-4093-a4d2-f63a966df637" (UID: "2bef9404-f32d-4093-a4d2-f63a966df637"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:00:46.213504 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.213479    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-workload-certs" pod "2bef9404-f32d-4093-a4d2-f63a966df637" (UID: "2bef9404-f32d-4093-a4d2-f63a966df637"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:00:46.215631 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.215575    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bef9404-f32d-4093-a4d2-f63a966df637-istio-token" pod "2bef9404-f32d-4093-a4d2-f63a966df637" (UID: "2bef9404-f32d-4093-a4d2-f63a966df637"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:00:46.215889 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.215869    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-istio-envoy" pod "2bef9404-f32d-4093-a4d2-f63a966df637" (UID: "2bef9404-f32d-4093-a4d2-f63a966df637"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:00:46.216071 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.216046    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bef9404-f32d-4093-a4d2-f63a966df637-kube-api-access-b5756" pod "2bef9404-f32d-4093-a4d2-f63a966df637" (UID: "2bef9404-f32d-4093-a4d2-f63a966df637"). InnerVolumeSpecName "kube-api-access-b5756". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:00:46.221748 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.221719    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2bef9404-f32d-4093-a4d2-f63a966df637-istio-podinfo" pod "2bef9404-f32d-4093-a4d2-f63a966df637" (UID: "2bef9404-f32d-4093-a4d2-f63a966df637"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:00:46.314233 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.314197    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5756\" (UniqueName: \"kubernetes.io/projected/2bef9404-f32d-4093-a4d2-f63a966df637-kube-api-access-b5756\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:46.314233 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.314228    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:46.314233 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.314238    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bef9404-f32d-4093-a4d2-f63a966df637-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:46.314517 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.314246    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:46.314517 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.314255    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bef9404-f32d-4093-a4d2-f63a966df637-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:46.314517 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.314263    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:46.314517 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.314272    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:46.314517 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.314281    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bef9404-f32d-4093-a4d2-f63a966df637-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:46.314517 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.314289    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bef9404-f32d-4093-a4d2-f63a966df637-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:00:46.356085 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.356054    4945 generic.go:356] "Generic (PLEG): container finished" podID="2bef9404-f32d-4093-a4d2-f63a966df637" containerID="5145345ea68150b357904a5f2f1b906ce1bd99c0e2c8a316f5900be0bd0662b2" exitCode=0
Oct 30 13:00:46.356269 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.356101    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"
Oct 30 13:00:46.356269 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.356136    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m" event={"ID":"2bef9404-f32d-4093-a4d2-f63a966df637","Type":"ContainerDied","Data":"5145345ea68150b357904a5f2f1b906ce1bd99c0e2c8a316f5900be0bd0662b2"}
Oct 30 13:00:46.356269 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.356189    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m" event={"ID":"2bef9404-f32d-4093-a4d2-f63a966df637","Type":"ContainerDied","Data":"be1c9edb234ff7fc3274fdedb2c593b565ba33af6d4dc2667a685504ad70ac6b"}
Oct 30 13:00:46.356269 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.356209    4945 scope.go:122] "RemoveContainer" containerID="5145345ea68150b357904a5f2f1b906ce1bd99c0e2c8a316f5900be0bd0662b2"
Oct 30 13:00:46.368282 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.368267    4945 scope.go:122] "RemoveContainer" containerID="5145345ea68150b357904a5f2f1b906ce1bd99c0e2c8a316f5900be0bd0662b2"
Oct 30 13:00:46.368536 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:00:46.368520    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5145345ea68150b357904a5f2f1b906ce1bd99c0e2c8a316f5900be0bd0662b2\": container with ID starting with 5145345ea68150b357904a5f2f1b906ce1bd99c0e2c8a316f5900be0bd0662b2 not found: ID does not exist" containerID="5145345ea68150b357904a5f2f1b906ce1bd99c0e2c8a316f5900be0bd0662b2"
Oct 30 13:00:46.368613 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.368549    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5145345ea68150b357904a5f2f1b906ce1bd99c0e2c8a316f5900be0bd0662b2"} err="failed to get container status \"5145345ea68150b357904a5f2f1b906ce1bd99c0e2c8a316f5900be0bd0662b2\": rpc error: code = NotFound desc = could not find container \"5145345ea68150b357904a5f2f1b906ce1bd99c0e2c8a316f5900be0bd0662b2\": container with ID starting with 5145345ea68150b357904a5f2f1b906ce1bd99c0e2c8a316f5900be0bd0662b2 not found: ID does not exist"
Oct 30 13:00:46.429288 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.429213    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"]
Oct 30 13:00:46.434125 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:46.433881    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-2bfb1b-openshift-default-79d9495f97-fb57m"]
Oct 30 13:00:47.551316 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:47.551283    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2bef9404-f32d-4093-a4d2-f63a966df637" path="/var/lib/kubelet/pods/2bef9404-f32d-4093-a4d2-f63a966df637/volumes"
Oct 30 13:00:48.148622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.145567    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"]
Oct 30 13:00:48.148622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.146907    4945 cpu_manager.go:414] "containerMap: removing container" podUID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4" containerName="cleanup"
Oct 30 13:00:48.148622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.146930    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4" containerName="cleanup"
Oct 30 13:00:48.148622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.146963    4945 cpu_manager.go:414] "containerMap: removing container" podUID="2bef9404-f32d-4093-a4d2-f63a966df637" containerName="istio-proxy"
Oct 30 13:00:48.148622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.146971    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2bef9404-f32d-4093-a4d2-f63a966df637" containerName="istio-proxy"
Oct 30 13:00:48.148622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.147143    4945 cpu_manager.go:414] "containerMap: removing container" podUID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4" containerName="cleanup"
Oct 30 13:00:48.148622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.147155    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4" containerName="cleanup"
Oct 30 13:00:48.148622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.147183    4945 cpu_manager.go:414] "containerMap: removing container" podUID="8293650f-2035-41cb-8db9-7942342fd116" containerName="istio-proxy"
Oct 30 13:00:48.148622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.147190    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8293650f-2035-41cb-8db9-7942342fd116" containerName="istio-proxy"
Oct 30 13:00:48.148622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.147201    4945 cpu_manager.go:414] "containerMap: removing container" podUID="e9cf7ee9-33f1-4291-834b-8289c4e7b4e0" containerName="maas-api"
Oct 30 13:00:48.148622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.147208    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9cf7ee9-33f1-4291-834b-8289c4e7b4e0" containerName="maas-api"
Oct 30 13:00:48.149377 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.148969    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4" containerName="cleanup"
Oct 30 13:00:48.149377 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.148995    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="2bef9404-f32d-4093-a4d2-f63a966df637" containerName="istio-proxy"
Oct 30 13:00:48.149377 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.149014    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9cf7ee9-33f1-4291-834b-8289c4e7b4e0" containerName="maas-api"
Oct 30 13:00:48.149377 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.149027    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4" containerName="cleanup"
Oct 30 13:00:48.198668 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.195822    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"]
Oct 30 13:00:48.198668 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.195956    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:48.215636 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.214055    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-e74c27-serving-cert\""
Oct 30 13:00:48.244908 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.244875    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28e5ff40-671c-422e-9875-82ddc3c878ce-metrics-certs\") pod \"maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:48.245098 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.245084    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28e5ff40-671c-422e-9875-82ddc3c878ce-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:48.245225 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.245209    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n9tk\" (UniqueName: \"kubernetes.io/projected/28e5ff40-671c-422e-9875-82ddc3c878ce-kube-api-access-4n9tk\") pod \"maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:48.245327 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.245300    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28e5ff40-671c-422e-9875-82ddc3c878ce-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:48.346758 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.346680    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n9tk\" (UniqueName: \"kubernetes.io/projected/28e5ff40-671c-422e-9875-82ddc3c878ce-kube-api-access-4n9tk\") pod \"maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:48.346758 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.346734    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28e5ff40-671c-422e-9875-82ddc3c878ce-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:48.346965 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.346810    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28e5ff40-671c-422e-9875-82ddc3c878ce-metrics-certs\") pod \"maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:48.347028 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.346990    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28e5ff40-671c-422e-9875-82ddc3c878ce-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:48.347533 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.347507    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28e5ff40-671c-422e-9875-82ddc3c878ce-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:48.349379 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.349351    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28e5ff40-671c-422e-9875-82ddc3c878ce-maas-api-tls\") pod \"maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:48.349522 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.349497    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28e5ff40-671c-422e-9875-82ddc3c878ce-metrics-certs\") pod \"maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:48.365580 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.365557    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n9tk\" (UniqueName: \"kubernetes.io/projected/28e5ff40-671c-422e-9875-82ddc3c878ce-kube-api-access-4n9tk\") pod \"maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:48.513835 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.513784    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:48.860186 ip-10-0-142-36 kubenswrapper[4945]: W1030 13:00:48.860159    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28e5ff40_671c_422e_9875_82ddc3c878ce.slice/crio-8bfaa0c9f1252b1ecebb92050cc95b720a37238105856de07b5ea9abaf12e96b WatchSource:0}: Error finding container 8bfaa0c9f1252b1ecebb92050cc95b720a37238105856de07b5ea9abaf12e96b: Status 404 returned error can't find the container with id 8bfaa0c9f1252b1ecebb92050cc95b720a37238105856de07b5ea9abaf12e96b
Oct 30 13:00:48.860519 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:48.860427    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"]
Oct 30 13:00:49.411840 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:49.411804    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s" event={"ID":"28e5ff40-671c-422e-9875-82ddc3c878ce","Type":"ContainerStarted","Data":"577de9c838789d130ef3beaeef3040394b3919b5439ddc38012f5acbc55df172"}
Oct 30 13:00:49.412131 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:49.412098    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s" event={"ID":"28e5ff40-671c-422e-9875-82ddc3c878ce","Type":"ContainerStarted","Data":"8bfaa0c9f1252b1ecebb92050cc95b720a37238105856de07b5ea9abaf12e96b"}
Oct 30 13:00:49.412274 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:49.412252    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:49.450972 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:49.450911    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s" podStartSLOduration=1.078900793 podStartE2EDuration="1.450896728s" totalImagesPullingTime="371.995935ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:00:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:00:49.449257295 +0000 UTC m=+1494.027206384" watchObservedRunningTime="2026-10-30 13:00:49.450896728 +0000 UTC m=+1494.028845817"
Oct 30 13:00:55.432312 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:55.432282    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:00:55.880554 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:55.880522    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 13:00:55.905058 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:00:55.905026    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 13:00:56.202833 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:00:56.202788    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:01:01.720130 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:01.720097    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"]
Oct 30 13:01:01.720635 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:01.720360    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h" podUID="f2d5f437-5553-460a-b1f8-dc47da09a5a7" containerName="maas-api" containerID="cri-o://7426de4d078058cfc25d2e143c89ecfb79715bb7b043b6b68c7c592d00d95871" gracePeriod=30
Oct 30 13:01:02.100189 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.100160    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 13:01:02.284640 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.269302    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2d5f437-5553-460a-b1f8-dc47da09a5a7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2d5f437-5553-460a-b1f8-dc47da09a5a7-metrics-certs\") pod \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") "
Oct 30 13:01:02.284640 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.269380    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2d5f437-5553-460a-b1f8-dc47da09a5a7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2d5f437-5553-460a-b1f8-dc47da09a5a7-openshift-service-ca\") pod \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") "
Oct 30 13:01:02.284640 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.269685    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2d5f437-5553-460a-b1f8-dc47da09a5a7-kube-api-access-jtfqp\" (UniqueName: \"kubernetes.io/projected/f2d5f437-5553-460a-b1f8-dc47da09a5a7-kube-api-access-jtfqp\") pod \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") "
Oct 30 13:01:02.284640 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.269735    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f2d5f437-5553-460a-b1f8-dc47da09a5a7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2d5f437-5553-460a-b1f8-dc47da09a5a7-maas-api-tls\") pod \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\" (UID: \"f2d5f437-5553-460a-b1f8-dc47da09a5a7\") "
Oct 30 13:01:02.306676 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.304937    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2d5f437-5553-460a-b1f8-dc47da09a5a7-openshift-service-ca" pod "f2d5f437-5553-460a-b1f8-dc47da09a5a7" (UID: "f2d5f437-5553-460a-b1f8-dc47da09a5a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:01:02.321621 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.320133    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2d5f437-5553-460a-b1f8-dc47da09a5a7-maas-api-tls" pod "f2d5f437-5553-460a-b1f8-dc47da09a5a7" (UID: "f2d5f437-5553-460a-b1f8-dc47da09a5a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:01:02.338075 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.338032    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2d5f437-5553-460a-b1f8-dc47da09a5a7-kube-api-access-jtfqp" pod "f2d5f437-5553-460a-b1f8-dc47da09a5a7" (UID: "f2d5f437-5553-460a-b1f8-dc47da09a5a7"). InnerVolumeSpecName "kube-api-access-jtfqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:01:02.358624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.347704    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2d5f437-5553-460a-b1f8-dc47da09a5a7-metrics-certs" pod "f2d5f437-5553-460a-b1f8-dc47da09a5a7" (UID: "f2d5f437-5553-460a-b1f8-dc47da09a5a7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:01:02.376844 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.376806    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2d5f437-5553-460a-b1f8-dc47da09a5a7-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:01:02.376844 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.376846    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtfqp\" (UniqueName: \"kubernetes.io/projected/f2d5f437-5553-460a-b1f8-dc47da09a5a7-kube-api-access-jtfqp\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:01:02.377113 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.376862    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2d5f437-5553-460a-b1f8-dc47da09a5a7-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:01:02.377113 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.376876    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f2d5f437-5553-460a-b1f8-dc47da09a5a7-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:01:02.522746 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.522713    4945 generic.go:356] "Generic (PLEG): container finished" podID="f2d5f437-5553-460a-b1f8-dc47da09a5a7" containerID="7426de4d078058cfc25d2e143c89ecfb79715bb7b043b6b68c7c592d00d95871" exitCode=0
Oct 30 13:01:02.522953 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.522770    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"
Oct 30 13:01:02.522953 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.522797    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h" event={"ID":"f2d5f437-5553-460a-b1f8-dc47da09a5a7","Type":"ContainerDied","Data":"7426de4d078058cfc25d2e143c89ecfb79715bb7b043b6b68c7c592d00d95871"}
Oct 30 13:01:02.522953 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.522852    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h" event={"ID":"f2d5f437-5553-460a-b1f8-dc47da09a5a7","Type":"ContainerDied","Data":"b131fd4b37d3c9757c0129e49d82fdba8a1adc98b20d674ac8b3bd908ccbb44e"}
Oct 30 13:01:02.522953 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.522876    4945 scope.go:122] "RemoveContainer" containerID="7426de4d078058cfc25d2e143c89ecfb79715bb7b043b6b68c7c592d00d95871"
Oct 30 13:01:02.533518 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.533500    4945 scope.go:122] "RemoveContainer" containerID="7426de4d078058cfc25d2e143c89ecfb79715bb7b043b6b68c7c592d00d95871"
Oct 30 13:01:02.533792 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:01:02.533769    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7426de4d078058cfc25d2e143c89ecfb79715bb7b043b6b68c7c592d00d95871\": container with ID starting with 7426de4d078058cfc25d2e143c89ecfb79715bb7b043b6b68c7c592d00d95871 not found: ID does not exist" containerID="7426de4d078058cfc25d2e143c89ecfb79715bb7b043b6b68c7c592d00d95871"
Oct 30 13:01:02.533892 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.533798    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7426de4d078058cfc25d2e143c89ecfb79715bb7b043b6b68c7c592d00d95871"} err="failed to get container status \"7426de4d078058cfc25d2e143c89ecfb79715bb7b043b6b68c7c592d00d95871\": rpc error: code = NotFound desc = could not find container \"7426de4d078058cfc25d2e143c89ecfb79715bb7b043b6b68c7c592d00d95871\": container with ID starting with 7426de4d078058cfc25d2e143c89ecfb79715bb7b043b6b68c7c592d00d95871 not found: ID does not exist"
Oct 30 13:01:02.574448 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.574341    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"]
Oct 30 13:01:02.580119 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:02.580026    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f0a01c-74c7b7d4c-z969h"]
Oct 30 13:01:03.549640 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:03.549589    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2d5f437-5553-460a-b1f8-dc47da09a5a7" path="/var/lib/kubelet/pods/f2d5f437-5553-460a-b1f8-dc47da09a5a7/volumes"
Oct 30 13:01:26.197488 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:01:26.197440    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:01:35.436942 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.436657    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"]
Oct 30 13:01:35.437694 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.437667    4945 cpu_manager.go:414] "containerMap: removing container" podUID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4" containerName="cleanup"
Oct 30 13:01:35.437694 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.437694    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4" containerName="cleanup"
Oct 30 13:01:35.437876 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.437751    4945 cpu_manager.go:414] "containerMap: removing container" podUID="f2d5f437-5553-460a-b1f8-dc47da09a5a7" containerName="maas-api"
Oct 30 13:01:35.437876 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.437760    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2d5f437-5553-460a-b1f8-dc47da09a5a7" containerName="maas-api"
Oct 30 13:01:35.438024 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.438005    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b8ed252-f4f4-43d3-8f30-93aabe4a30b4" containerName="cleanup"
Oct 30 13:01:35.438090 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.438050    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2d5f437-5553-460a-b1f8-dc47da09a5a7" containerName="maas-api"
Oct 30 13:01:35.451172 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.449015    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"]
Oct 30 13:01:35.451172 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.449136    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"
Oct 30 13:01:35.480495 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.480446    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-e74c27\""
Oct 30 13:01:35.480812 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.480546    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-e74c27-dockercfg-l9fbx\""
Oct 30 13:01:35.604350 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.604312    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkb6s\" (UniqueName: \"kubernetes.io/projected/efb9145f-43ea-4eb9-9700-cd84fbfa8459-kube-api-access-jkb6s\") pod \"payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm\" (UID: \"efb9145f-43ea-4eb9-9700-cd84fbfa8459\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"
Oct 30 13:01:35.604642 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.604617    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efb9145f-43ea-4eb9-9700-cd84fbfa8459-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm\" (UID: \"efb9145f-43ea-4eb9-9700-cd84fbfa8459\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"
Oct 30 13:01:35.706180 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.706125    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efb9145f-43ea-4eb9-9700-cd84fbfa8459-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm\" (UID: \"efb9145f-43ea-4eb9-9700-cd84fbfa8459\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"
Oct 30 13:01:35.706386 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.706286    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkb6s\" (UniqueName: \"kubernetes.io/projected/efb9145f-43ea-4eb9-9700-cd84fbfa8459-kube-api-access-jkb6s\") pod \"payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm\" (UID: \"efb9145f-43ea-4eb9-9700-cd84fbfa8459\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"
Oct 30 13:01:35.707103 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.707078    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efb9145f-43ea-4eb9-9700-cd84fbfa8459-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm\" (UID: \"efb9145f-43ea-4eb9-9700-cd84fbfa8459\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"
Oct 30 13:01:35.725157 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.725129    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkb6s\" (UniqueName: \"kubernetes.io/projected/efb9145f-43ea-4eb9-9700-cd84fbfa8459-kube-api-access-jkb6s\") pod \"payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm\" (UID: \"efb9145f-43ea-4eb9-9700-cd84fbfa8459\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"
Oct 30 13:01:35.772995 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:35.772963    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"
Oct 30 13:01:36.080643 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:36.080519    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"]
Oct 30 13:01:36.084918 ip-10-0-142-36 kubenswrapper[4945]: W1030 13:01:36.084891    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb9145f_43ea_4eb9_9700_cd84fbfa8459.slice/crio-0fd10510ca355b6c3eedf45853e4d3cda4ae512e4e181f30f94233fe6c1b6b69 WatchSource:0}: Error finding container 0fd10510ca355b6c3eedf45853e4d3cda4ae512e4e181f30f94233fe6c1b6b69: Status 404 returned error can't find the container with id 0fd10510ca355b6c3eedf45853e4d3cda4ae512e4e181f30f94233fe6c1b6b69
Oct 30 13:01:36.907201 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:36.907159    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm" event={"ID":"efb9145f-43ea-4eb9-9700-cd84fbfa8459","Type":"ContainerStarted","Data":"0fd10510ca355b6c3eedf45853e4d3cda4ae512e4e181f30f94233fe6c1b6b69"}
Oct 30 13:01:37.677443 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.677368    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"]
Oct 30 13:01:37.682080 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.682059    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:37.700171 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.700142    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"]
Oct 30 13:01:37.706714 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.706680    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-87f1b5-serving-cert\""
Oct 30 13:01:37.841779 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.841740    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtwfd\" (UniqueName: \"kubernetes.io/projected/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-kube-api-access-jtwfd\") pod \"maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:37.841952 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.841802    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-metrics-certs\") pod \"maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:37.842012 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.841989    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-maas-api-tls\") pod \"maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:37.842049 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.842028    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:37.942746 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.942638    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm" event={"ID":"efb9145f-43ea-4eb9-9700-cd84fbfa8459","Type":"ContainerStarted","Data":"72f2e0f735e42723deaaa769f0d5056b6c221257afbde97cccae8dc44a012a7c"}
Oct 30 13:01:37.943785 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.943730    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"
Oct 30 13:01:37.944466 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.944444    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-maas-api-tls\") pod \"maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:37.944646 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.944480    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:37.944646 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.944579    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtwfd\" (UniqueName: \"kubernetes.io/projected/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-kube-api-access-jtwfd\") pod \"maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:37.944761 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.944663    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-metrics-certs\") pod \"maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:37.945522 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.945496    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:37.947259 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.947236    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-maas-api-tls\") pod \"maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:37.947338 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.947287    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-metrics-certs\") pod \"maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:37.964130 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.964099    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtwfd\" (UniqueName: \"kubernetes.io/projected/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-kube-api-access-jtwfd\") pod \"maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:37.990395 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.990312    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm" podStartSLOduration=1.810156876 podStartE2EDuration="2.990289168s" totalImagesPullingTime="1.180132292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:01:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:01:37.980120874 +0000 UTC m=+1542.558069962" watchObservedRunningTime="2026-10-30 13:01:37.990289168 +0000 UTC m=+1542.568238259"
Oct 30 13:01:37.996786 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:37.996752    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:38.319530 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:38.319503    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"]
Oct 30 13:01:38.322653 ip-10-0-142-36 kubenswrapper[4945]: W1030 13:01:38.322618    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fcf4e3a_dfbd_4ceb_bd7f_2e98fd2ba0b7.slice/crio-7d4d785b2c6e5ed391af3d52d2f3b134e18211764dadf8d518efbdb6027ac47a WatchSource:0}: Error finding container 7d4d785b2c6e5ed391af3d52d2f3b134e18211764dadf8d518efbdb6027ac47a: Status 404 returned error can't find the container with id 7d4d785b2c6e5ed391af3d52d2f3b134e18211764dadf8d518efbdb6027ac47a
Oct 30 13:01:38.954495 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:38.953187    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz" event={"ID":"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7","Type":"ContainerStarted","Data":"390d6575c7e80eb7ce57bf4394e5ea66b0861c1f67469c4050f235b7dfa515cf"}
Oct 30 13:01:38.956307 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:38.954530    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:38.956307 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:38.954562    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz" event={"ID":"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7","Type":"ContainerStarted","Data":"7d4d785b2c6e5ed391af3d52d2f3b134e18211764dadf8d518efbdb6027ac47a"}
Oct 30 13:01:38.995206 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:38.995104    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz" podStartSLOduration=1.575042622 podStartE2EDuration="1.995090302s" totalImagesPullingTime="420.04768ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:01:38.992495536 +0000 UTC m=+1543.570444626" watchObservedRunningTime="2026-10-30 13:01:38.995090302 +0000 UTC m=+1543.573039391"
Oct 30 13:01:40.974206 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:40.974150    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"]
Oct 30 13:01:40.974821 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:40.974428    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r" podUID="a39c4f07-8275-4aff-8cb5-68390f8ed2ad" containerName="maas-api" containerID="cri-o://a49c80c7e0152733f4030e833d33538b9c36458699c8aac243835bff519a7ce3" gracePeriod=30
Oct 30 13:01:41.375198 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.375164    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 13:01:41.437426 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.437392    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-openshift-service-ca\") pod \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") "
Oct 30 13:01:41.437655 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.437536    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-maas-api-tls\") pod \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") "
Oct 30 13:01:41.437655 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.437634    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-kube-api-access-58nmd\" (UniqueName: \"kubernetes.io/projected/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-kube-api-access-58nmd\") pod \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") "
Oct 30 13:01:41.437786 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.437675    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-metrics-certs\") pod \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\" (UID: \"a39c4f07-8275-4aff-8cb5-68390f8ed2ad\") "
Oct 30 13:01:41.438350 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.438306    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-openshift-service-ca" pod "a39c4f07-8275-4aff-8cb5-68390f8ed2ad" (UID: "a39c4f07-8275-4aff-8cb5-68390f8ed2ad"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:01:41.441882 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.440956    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-metrics-certs" pod "a39c4f07-8275-4aff-8cb5-68390f8ed2ad" (UID: "a39c4f07-8275-4aff-8cb5-68390f8ed2ad"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:01:41.441882 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.441013    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-maas-api-tls" pod "a39c4f07-8275-4aff-8cb5-68390f8ed2ad" (UID: "a39c4f07-8275-4aff-8cb5-68390f8ed2ad"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:01:41.442856 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.442831    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-kube-api-access-58nmd" pod "a39c4f07-8275-4aff-8cb5-68390f8ed2ad" (UID: "a39c4f07-8275-4aff-8cb5-68390f8ed2ad"). InnerVolumeSpecName "kube-api-access-58nmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:01:41.542446 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.542402    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:01:41.542446 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.542445    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:01:41.542749 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.542461    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58nmd\" (UniqueName: \"kubernetes.io/projected/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-kube-api-access-58nmd\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:01:41.542749 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.542477    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a39c4f07-8275-4aff-8cb5-68390f8ed2ad-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:01:41.828637 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.828561    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"]
Oct 30 13:01:41.829631 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.829576    4945 cpu_manager.go:414] "containerMap: removing container" podUID="a39c4f07-8275-4aff-8cb5-68390f8ed2ad" containerName="maas-api"
Oct 30 13:01:41.829631 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.829617    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a39c4f07-8275-4aff-8cb5-68390f8ed2ad" containerName="maas-api"
Oct 30 13:01:41.829809 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.829794    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="a39c4f07-8275-4aff-8cb5-68390f8ed2ad" containerName="maas-api"
Oct 30 13:01:41.839065 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.839040    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:41.854381 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.854354    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-a81a48-serving-cert\""
Oct 30 13:01:41.855929 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.855439    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6ade70c-d9ac-4854-baae-0190ebd5ae08-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:41.855929 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.855537    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6ade70c-d9ac-4854-baae-0190ebd5ae08-metrics-certs\") pod \"maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:41.855929 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.855589    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6ade70c-d9ac-4854-baae-0190ebd5ae08-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:41.855929 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.855679    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mk9z\" (UniqueName: \"kubernetes.io/projected/a6ade70c-d9ac-4854-baae-0190ebd5ae08-kube-api-access-9mk9z\") pod \"maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:41.861593 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.861364    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"]
Oct 30 13:01:41.957424 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.957337    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6ade70c-d9ac-4854-baae-0190ebd5ae08-metrics-certs\") pod \"maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:41.957424 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.957382    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6ade70c-d9ac-4854-baae-0190ebd5ae08-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:41.957424 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.957424    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mk9z\" (UniqueName: \"kubernetes.io/projected/a6ade70c-d9ac-4854-baae-0190ebd5ae08-kube-api-access-9mk9z\") pod \"maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:41.957762 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.957667    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6ade70c-d9ac-4854-baae-0190ebd5ae08-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:41.958180 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.958146    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6ade70c-d9ac-4854-baae-0190ebd5ae08-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:41.960031 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.960010    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6ade70c-d9ac-4854-baae-0190ebd5ae08-metrics-certs\") pod \"maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:41.960134 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.960101    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6ade70c-d9ac-4854-baae-0190ebd5ae08-maas-api-tls\") pod \"maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:41.975861 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.975836    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mk9z\" (UniqueName: \"kubernetes.io/projected/a6ade70c-d9ac-4854-baae-0190ebd5ae08-kube-api-access-9mk9z\") pod \"maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:41.979001 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.978977    4945 generic.go:356] "Generic (PLEG): container finished" podID="a39c4f07-8275-4aff-8cb5-68390f8ed2ad" containerID="a49c80c7e0152733f4030e833d33538b9c36458699c8aac243835bff519a7ce3" exitCode=0
Oct 30 13:01:41.979122 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.979033    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r" event={"ID":"a39c4f07-8275-4aff-8cb5-68390f8ed2ad","Type":"ContainerDied","Data":"a49c80c7e0152733f4030e833d33538b9c36458699c8aac243835bff519a7ce3"}
Oct 30 13:01:41.979122 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.979039    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"
Oct 30 13:01:41.979122 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.979074    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r" event={"ID":"a39c4f07-8275-4aff-8cb5-68390f8ed2ad","Type":"ContainerDied","Data":"dd64e6a87906aab3c2ce6af8f20d5c71ad4cd6eb144bd59c99fb4f65600f9b79"}
Oct 30 13:01:41.979122 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:41.979099    4945 scope.go:122] "RemoveContainer" containerID="a49c80c7e0152733f4030e833d33538b9c36458699c8aac243835bff519a7ce3"
Oct 30 13:01:42.007421 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:42.007398    4945 scope.go:122] "RemoveContainer" containerID="a49c80c7e0152733f4030e833d33538b9c36458699c8aac243835bff519a7ce3"
Oct 30 13:01:42.007878 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:01:42.007851    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a49c80c7e0152733f4030e833d33538b9c36458699c8aac243835bff519a7ce3\": container with ID starting with a49c80c7e0152733f4030e833d33538b9c36458699c8aac243835bff519a7ce3 not found: ID does not exist" containerID="a49c80c7e0152733f4030e833d33538b9c36458699c8aac243835bff519a7ce3"
Oct 30 13:01:42.007955 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:42.007890    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a49c80c7e0152733f4030e833d33538b9c36458699c8aac243835bff519a7ce3"} err="failed to get container status \"a49c80c7e0152733f4030e833d33538b9c36458699c8aac243835bff519a7ce3\": rpc error: code = NotFound desc = could not find container \"a49c80c7e0152733f4030e833d33538b9c36458699c8aac243835bff519a7ce3\": container with ID starting with a49c80c7e0152733f4030e833d33538b9c36458699c8aac243835bff519a7ce3 not found: ID does not exist"
Oct 30 13:01:42.048652 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:42.047680    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"]
Oct 30 13:01:42.052908 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:42.052770    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f1222e-7f559cb5bc-fq77r"]
Oct 30 13:01:42.182172 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:42.182132    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:42.495995 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:42.495967    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"]
Oct 30 13:01:42.498339 ip-10-0-142-36 kubenswrapper[4945]: W1030 13:01:42.498307    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6ade70c_d9ac_4854_baae_0190ebd5ae08.slice/crio-c72907e67d0abf954e0fe03f7342bebe07867289c4137d6d932d2254a36152f2 WatchSource:0}: Error finding container c72907e67d0abf954e0fe03f7342bebe07867289c4137d6d932d2254a36152f2: Status 404 returned error can't find the container with id c72907e67d0abf954e0fe03f7342bebe07867289c4137d6d932d2254a36152f2
Oct 30 13:01:42.988919 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:42.988881    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw" event={"ID":"a6ade70c-d9ac-4854-baae-0190ebd5ae08","Type":"ContainerStarted","Data":"cc253c42d801b0d0b669221426b4ad43b04dd7998abcca9e9a212d452b871a23"}
Oct 30 13:01:42.989395 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:42.988934    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw" event={"ID":"a6ade70c-d9ac-4854-baae-0190ebd5ae08","Type":"ContainerStarted","Data":"c72907e67d0abf954e0fe03f7342bebe07867289c4137d6d932d2254a36152f2"}
Oct 30 13:01:42.989395 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:42.988965    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:43.031666 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:43.031593    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw" podStartSLOduration=1.708193839 podStartE2EDuration="2.031577327s" totalImagesPullingTime="323.383488ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:01:43.027626437 +0000 UTC m=+1547.605575523" watchObservedRunningTime="2026-10-30 13:01:43.031577327 +0000 UTC m=+1547.609526424"
Oct 30 13:01:43.551179 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:43.551140    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a39c4f07-8275-4aff-8cb5-68390f8ed2ad" path="/var/lib/kubelet/pods/a39c4f07-8275-4aff-8cb5-68390f8ed2ad/volumes"
Oct 30 13:01:44.992752 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:44.992716    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:01:49.005382 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:49.005350    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:01:49.964542 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:01:49.964510    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"
Oct 30 13:01:56.200262 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:01:56.200219    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:02:01.755234 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:01.754718    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:02:01.755234 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:01.754797    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:01.778712 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:01.778675    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:02:01.778971 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:01.778944    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:01.841774 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:01.841039    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:01.898621 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:01.896359    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:01.898932 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:01.898903    4945 patch_prober.go:29] interesting pod/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:02:01.899105 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:01.899080    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:01.965829 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:01.965777    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:02.067673 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:02.053692    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:02.067673 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:02.053941    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 13:02:02.087622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:02.075712    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:02.921624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:02.899694    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:02.921624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:02.901104    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": context deadline exceeded"
Oct 30 13:02:02.921624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:02.901220    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 13:02:02.934631 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:02.931730    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:02.972622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:02.972100    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:02.992623 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:02.982724    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 13:02:03.089819 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:03.089766    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:02:03.090117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:03.090085    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 13:02:03.090723 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:03.090698    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 13:02:03.090831 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:03.090765    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 13:02:03.400723 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:03.397726    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 13:02:03.916626 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:03.870936    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 13:02:03.946749 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:03.904967    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 13:02:03.975860 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:03.975813    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:04.190904 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:04.190783    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 13:02:04.191538 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:04.190697    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:04.350362 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:04.350317    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:04.369667 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:04.352441    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:02:04.438369 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:04.402520    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 13:02:04.505630 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:04.475902    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 13:02:04.506045 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:04.506016    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 13:02:05.141636 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:05.129693    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 13:02:05.141636 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:05.134693    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 13:02:05.171125 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:05.170704    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:05.240727 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:05.212196    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 13:02:05.281625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:05.248721    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:05.375158 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:05.375098    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 13:02:05.420782 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:05.420241    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:02:05.420782 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:05.420308    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt" podUID="579ade75-eb45-4c2b-add1-d199586065c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:05.439419 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:05.439370    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:05.439931 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:05.439899    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:02:05.609624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:05.580111    4945 patch_prober.go:29] interesting pod/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:02:05.609624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:05.580198    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr" podUID="facb034f-f30c-4290-8ad3-07b58b7163e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:06.130624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.126722    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:02:06.130624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.126799    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:06.130624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.127722    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:02:06.130624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.127788    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:06.134618 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.131663    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 13:02:06.175656 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.172916    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 13:02:06.220622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.218785    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:06.268624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.268162    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:06.405177 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.405051    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:06.456726 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.455768    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:06.456726 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.456166    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 13:02:06.765757 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.764863    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 13:02:06.815139 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.806041    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 13:02:06.815139 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.807147    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 13:02:06.965748 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.965711    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:02:06.966032 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:06.965991    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" podUID="a069f35f-76b4-4583-857a-304896b3f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:07.135276 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:07.134683    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:07.275625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:07.272723    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 13:02:07.459320 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:07.459186    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:02:07.459730 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:07.459645    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 13:02:07.941543 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:07.936884    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 13:02:08.261719 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:08.258703    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:02:08.309019 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:08.307225    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 13:02:08.332313 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:08.331093    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 13:02:08.604682 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:08.601423    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:02:08.853975 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:08.853937    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 13:02:09.049624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:09.030139    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 13:02:21.265273 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.265222    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"]
Oct 30 13:02:21.275665 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.275635    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.305724 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.305676    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"]
Oct 30 13:02:21.366262 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.366230    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-096cc3-openshift-default-dockercfg-5rpdm\""
Oct 30 13:02:21.435448 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.435417    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-workload-socket\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.435634 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.435549    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c73e1c53-dd91-4247-8f72-390a6708190d-istio-podinfo\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.435634 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.435585    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c73e1c53-dd91-4247-8f72-390a6708190d-istio-token\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.435759 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.435637    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-istio-data\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.435816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.435771    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fbrm\" (UniqueName: \"kubernetes.io/projected/c73e1c53-dd91-4247-8f72-390a6708190d-kube-api-access-8fbrm\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.435876 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.435826    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-credential-socket\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.435933 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.435899    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-istio-envoy\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.436037 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.436022    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c73e1c53-dd91-4247-8f72-390a6708190d-istiod-ca-cert\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.436104 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.436070    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-workload-certs\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.537622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.537536    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-istio-envoy\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.537770 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.537659    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c73e1c53-dd91-4247-8f72-390a6708190d-istiod-ca-cert\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.537770 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.537695    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-workload-certs\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.537770 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.537740    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-workload-socket\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.537911 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.537879    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c73e1c53-dd91-4247-8f72-390a6708190d-istio-podinfo\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.537967 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.537929    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c73e1c53-dd91-4247-8f72-390a6708190d-istio-token\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.537967 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.537957    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-istio-data\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.538088 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.538003    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fbrm\" (UniqueName: \"kubernetes.io/projected/c73e1c53-dd91-4247-8f72-390a6708190d-kube-api-access-8fbrm\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.538088 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.538066    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-credential-socket\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.538499 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.538236    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-workload-socket\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.538499 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.538396    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-istio-data\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.538499 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.538468    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-credential-socket\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.538759 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.538500    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c73e1c53-dd91-4247-8f72-390a6708190d-istiod-ca-cert\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.538759 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.538666    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-workload-certs\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.540707 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.540681    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-istio-envoy\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.540945 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.540899    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c73e1c53-dd91-4247-8f72-390a6708190d-istio-podinfo\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.556914 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.556890    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c73e1c53-dd91-4247-8f72-390a6708190d-istio-token\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.557188 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.557166    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fbrm\" (UniqueName: \"kubernetes.io/projected/c73e1c53-dd91-4247-8f72-390a6708190d-kube-api-access-8fbrm\") pod \"e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") " pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.624133 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.624097    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:21.945565 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.945535    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"]
Oct 30 13:02:21.946825 ip-10-0-142-36 kubenswrapper[4945]: W1030 13:02:21.946792    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73e1c53_dd91_4247_8f72_390a6708190d.slice/crio-44123cb74e348946885c3169f2b070a1baac6cebc6754e7b0b7c094c647dfc53 WatchSource:0}: Error finding container 44123cb74e348946885c3169f2b070a1baac6cebc6754e7b0b7c094c647dfc53: Status 404 returned error can't find the container with id 44123cb74e348946885c3169f2b070a1baac6cebc6754e7b0b7c094c647dfc53
Oct 30 13:02:21.949405 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.949371    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:02:21.949487 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.949440    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:02:21.949487 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:21.949471    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:02:22.516425 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:22.516390    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"]
Oct 30 13:02:22.517408 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:22.517365    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7" podUID="2000e193-dc40-4807-867f-48bad0847b73" containerName="maas-api" containerID="cri-o://3255a26944a66b5589c7a7e39c88099b6526d496ac562f2d77766d2430a3b16a" gracePeriod=30
Oct 30 13:02:22.957622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:22.954038    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr" event={"ID":"c73e1c53-dd91-4247-8f72-390a6708190d","Type":"ContainerStarted","Data":"49671fad8fbfc6059e731d437c5a94b9a3879c9b7f18a03ed49acba5edbaab2a"}
Oct 30 13:02:22.957622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:22.954146    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr" event={"ID":"c73e1c53-dd91-4247-8f72-390a6708190d","Type":"ContainerStarted","Data":"44123cb74e348946885c3169f2b070a1baac6cebc6754e7b0b7c094c647dfc53"}
Oct 30 13:02:22.958215 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:22.958182    4945 generic.go:356] "Generic (PLEG): container finished" podID="2000e193-dc40-4807-867f-48bad0847b73" containerID="3255a26944a66b5589c7a7e39c88099b6526d496ac562f2d77766d2430a3b16a" exitCode=0
Oct 30 13:02:22.959207 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:22.958670    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7" event={"ID":"2000e193-dc40-4807-867f-48bad0847b73","Type":"ContainerDied","Data":"3255a26944a66b5589c7a7e39c88099b6526d496ac562f2d77766d2430a3b16a"}
Oct 30 13:02:23.014133 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.014072    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr" podStartSLOduration=2.014050559 podStartE2EDuration="2.014050559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:02:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:02:23.002707984 +0000 UTC m=+1587.580657074" watchObservedRunningTime="2026-10-30 13:02:23.014050559 +0000 UTC m=+1587.591999650"
Oct 30 13:02:23.057181 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.057138    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"]
Oct 30 13:02:23.057985 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.057956    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s" podUID="28e5ff40-671c-422e-9875-82ddc3c878ce" containerName="maas-api" containerID="cri-o://577de9c838789d130ef3beaeef3040394b3919b5439ddc38012f5acbc55df172" gracePeriod=30
Oct 30 13:02:23.238717 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.227391    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:02:23.458475 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.401030    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2000e193-dc40-4807-867f-48bad0847b73-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2000e193-dc40-4807-867f-48bad0847b73-openshift-service-ca\") pod \"2000e193-dc40-4807-867f-48bad0847b73\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") "
Oct 30 13:02:23.458475 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.401107    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2000e193-dc40-4807-867f-48bad0847b73-kube-api-access-52lw7\" (UniqueName: \"kubernetes.io/projected/2000e193-dc40-4807-867f-48bad0847b73-kube-api-access-52lw7\") pod \"2000e193-dc40-4807-867f-48bad0847b73\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") "
Oct 30 13:02:23.458475 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.401414    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-maas-api-tls\") pod \"2000e193-dc40-4807-867f-48bad0847b73\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") "
Oct 30 13:02:23.458475 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.401488    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-metrics-certs\") pod \"2000e193-dc40-4807-867f-48bad0847b73\" (UID: \"2000e193-dc40-4807-867f-48bad0847b73\") "
Oct 30 13:02:23.458475 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.415954    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-metrics-certs" pod "2000e193-dc40-4807-867f-48bad0847b73" (UID: "2000e193-dc40-4807-867f-48bad0847b73"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:02:23.458475 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.417845    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2000e193-dc40-4807-867f-48bad0847b73-openshift-service-ca" pod "2000e193-dc40-4807-867f-48bad0847b73" (UID: "2000e193-dc40-4807-867f-48bad0847b73"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:02:23.458475 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.423757    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2000e193-dc40-4807-867f-48bad0847b73-kube-api-access-52lw7" pod "2000e193-dc40-4807-867f-48bad0847b73" (UID: "2000e193-dc40-4807-867f-48bad0847b73"). InnerVolumeSpecName "kube-api-access-52lw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:02:23.458475 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.426834    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-maas-api-tls" pod "2000e193-dc40-4807-867f-48bad0847b73" (UID: "2000e193-dc40-4807-867f-48bad0847b73"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:02:23.652545 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.605308    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:23.652545 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.605350    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2000e193-dc40-4807-867f-48bad0847b73-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:23.652545 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.605366    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2000e193-dc40-4807-867f-48bad0847b73-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:23.652545 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.605383    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52lw7\" (UniqueName: \"kubernetes.io/projected/2000e193-dc40-4807-867f-48bad0847b73-kube-api-access-52lw7\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:23.661660 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.657015    4945 patch_prober.go:29] interesting pod/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 13:02:23.661660 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.657070    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr" podUID="c73e1c53-dd91-4247-8f72-390a6708190d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 13:02:23.668752 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.668726    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:23.668892 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.668778    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"]
Oct 30 13:02:23.668956 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:23.668941    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm" podUID="efb9145f-43ea-4eb9-9700-cd84fbfa8459" containerName="payload-pre-processing" containerID="cri-o://72f2e0f735e42723deaaa769f0d5056b6c221257afbde97cccae8dc44a012a7c" gracePeriod=30
Oct 30 13:02:24.153895 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.152509    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7" event={"ID":"2000e193-dc40-4807-867f-48bad0847b73","Type":"ContainerDied","Data":"399e198c3263632422c17e681f65918998311bf23b726cb2feed527f2e3cb752"}
Oct 30 13:02:24.153895 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.152583    4945 scope.go:122] "RemoveContainer" containerID="3255a26944a66b5589c7a7e39c88099b6526d496ac562f2d77766d2430a3b16a"
Oct 30 13:02:24.153895 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.152770    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"
Oct 30 13:02:24.161743 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.160187    4945 generic.go:356] "Generic (PLEG): container finished" podID="28e5ff40-671c-422e-9875-82ddc3c878ce" containerID="577de9c838789d130ef3beaeef3040394b3919b5439ddc38012f5acbc55df172" exitCode=0
Oct 30 13:02:24.161743 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.160297    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s" event={"ID":"28e5ff40-671c-422e-9875-82ddc3c878ce","Type":"ContainerDied","Data":"577de9c838789d130ef3beaeef3040394b3919b5439ddc38012f5acbc55df172"}
Oct 30 13:02:24.162627 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.161920    4945 generic.go:356] "Generic (PLEG): container finished" podID="efb9145f-43ea-4eb9-9700-cd84fbfa8459" containerID="72f2e0f735e42723deaaa769f0d5056b6c221257afbde97cccae8dc44a012a7c" exitCode=0
Oct 30 13:02:24.162627 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.161987    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm" event={"ID":"efb9145f-43ea-4eb9-9700-cd84fbfa8459","Type":"ContainerDied","Data":"72f2e0f735e42723deaaa769f0d5056b6c221257afbde97cccae8dc44a012a7c"}
Oct 30 13:02:24.264906 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.264880    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:02:24.309922 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.288556    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"]
Oct 30 13:02:24.329939 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.329900    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-096cc3-556b449bc8-xw9k7"]
Oct 30 13:02:24.348976 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.348944    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/28e5ff40-671c-422e-9875-82ddc3c878ce-metrics-certs\" (UniqueName: \"kubernetes.io/secret/28e5ff40-671c-422e-9875-82ddc3c878ce-metrics-certs\") pod \"28e5ff40-671c-422e-9875-82ddc3c878ce\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") "
Oct 30 13:02:24.349108 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.348990    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28e5ff40-671c-422e-9875-82ddc3c878ce-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28e5ff40-671c-422e-9875-82ddc3c878ce-openshift-service-ca\") pod \"28e5ff40-671c-422e-9875-82ddc3c878ce\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") "
Oct 30 13:02:24.349108 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.349028    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/28e5ff40-671c-422e-9875-82ddc3c878ce-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28e5ff40-671c-422e-9875-82ddc3c878ce-maas-api-tls\") pod \"28e5ff40-671c-422e-9875-82ddc3c878ce\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") "
Oct 30 13:02:24.349507 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.349232    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28e5ff40-671c-422e-9875-82ddc3c878ce-kube-api-access-4n9tk\" (UniqueName: \"kubernetes.io/projected/28e5ff40-671c-422e-9875-82ddc3c878ce-kube-api-access-4n9tk\") pod \"28e5ff40-671c-422e-9875-82ddc3c878ce\" (UID: \"28e5ff40-671c-422e-9875-82ddc3c878ce\") "
Oct 30 13:02:24.350680 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.350652    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28e5ff40-671c-422e-9875-82ddc3c878ce-openshift-service-ca" pod "28e5ff40-671c-422e-9875-82ddc3c878ce" (UID: "28e5ff40-671c-422e-9875-82ddc3c878ce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:02:24.351830 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.351807    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28e5ff40-671c-422e-9875-82ddc3c878ce-metrics-certs" pod "28e5ff40-671c-422e-9875-82ddc3c878ce" (UID: "28e5ff40-671c-422e-9875-82ddc3c878ce"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:02:24.352665 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.352635    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28e5ff40-671c-422e-9875-82ddc3c878ce-maas-api-tls" pod "28e5ff40-671c-422e-9875-82ddc3c878ce" (UID: "28e5ff40-671c-422e-9875-82ddc3c878ce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:02:24.355710 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.355687    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28e5ff40-671c-422e-9875-82ddc3c878ce-kube-api-access-4n9tk" pod "28e5ff40-671c-422e-9875-82ddc3c878ce" (UID: "28e5ff40-671c-422e-9875-82ddc3c878ce"). InnerVolumeSpecName "kube-api-access-4n9tk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:02:24.439398 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.439369    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"
Oct 30 13:02:24.451708 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.451682    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efb9145f-43ea-4eb9-9700-cd84fbfa8459-kube-api-access-jkb6s\" (UniqueName: \"kubernetes.io/projected/efb9145f-43ea-4eb9-9700-cd84fbfa8459-kube-api-access-jkb6s\") pod \"efb9145f-43ea-4eb9-9700-cd84fbfa8459\" (UID: \"efb9145f-43ea-4eb9-9700-cd84fbfa8459\") "
Oct 30 13:02:24.451963 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.451948    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/efb9145f-43ea-4eb9-9700-cd84fbfa8459-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efb9145f-43ea-4eb9-9700-cd84fbfa8459-plugins-config-volume\") pod \"efb9145f-43ea-4eb9-9700-cd84fbfa8459\" (UID: \"efb9145f-43ea-4eb9-9700-cd84fbfa8459\") "
Oct 30 13:02:24.452319 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.452301    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28e5ff40-671c-422e-9875-82ddc3c878ce-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:24.452319 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.452309    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efb9145f-43ea-4eb9-9700-cd84fbfa8459-plugins-config-volume" pod "efb9145f-43ea-4eb9-9700-cd84fbfa8459" (UID: "efb9145f-43ea-4eb9-9700-cd84fbfa8459"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:02:24.452437 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.452323    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28e5ff40-671c-422e-9875-82ddc3c878ce-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:24.452437 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.452338    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28e5ff40-671c-422e-9875-82ddc3c878ce-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:24.452437 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.452352    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n9tk\" (UniqueName: \"kubernetes.io/projected/28e5ff40-671c-422e-9875-82ddc3c878ce-kube-api-access-4n9tk\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:24.453907 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.453884    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efb9145f-43ea-4eb9-9700-cd84fbfa8459-kube-api-access-jkb6s" pod "efb9145f-43ea-4eb9-9700-cd84fbfa8459" (UID: "efb9145f-43ea-4eb9-9700-cd84fbfa8459"). InnerVolumeSpecName "kube-api-access-jkb6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:02:24.554332 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.554287    4945 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/efb9145f-43ea-4eb9-9700-cd84fbfa8459-plugins-config-volume\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:24.554332 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.554320    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkb6s\" (UniqueName: \"kubernetes.io/projected/efb9145f-43ea-4eb9-9700-cd84fbfa8459-kube-api-access-jkb6s\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:24.625317 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.625274    4945 patch_prober.go:29] interesting pod/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 13:02:24.625492 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:24.625343    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr" podUID="c73e1c53-dd91-4247-8f72-390a6708190d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 13:02:25.193673 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:25.193000    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s" event={"ID":"28e5ff40-671c-422e-9875-82ddc3c878ce","Type":"ContainerDied","Data":"8bfaa0c9f1252b1ecebb92050cc95b720a37238105856de07b5ea9abaf12e96b"}
Oct 30 13:02:25.193673 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:25.193076    4945 scope.go:122] "RemoveContainer" containerID="577de9c838789d130ef3beaeef3040394b3919b5439ddc38012f5acbc55df172"
Oct 30 13:02:25.193673 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:25.193180    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"
Oct 30 13:02:25.224816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:25.224791    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"
Oct 30 13:02:25.226836 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:25.226781    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm" event={"ID":"efb9145f-43ea-4eb9-9700-cd84fbfa8459","Type":"ContainerDied","Data":"0fd10510ca355b6c3eedf45853e4d3cda4ae512e4e181f30f94233fe6c1b6b69"}
Oct 30 13:02:25.244833 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:25.244812    4945 scope.go:122] "RemoveContainer" containerID="72f2e0f735e42723deaaa769f0d5056b6c221257afbde97cccae8dc44a012a7c"
Oct 30 13:02:25.269437 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:25.269414    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"]
Oct 30 13:02:25.311529 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:25.311246    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-e74c27-8dcc94b94-nmf4s"]
Oct 30 13:02:25.331943 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:25.331901    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"]
Oct 30 13:02:25.335999 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:25.335976    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-e74c27-79f49467d7-ttpbm"]
Oct 30 13:02:25.560417 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:25.560336    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2000e193-dc40-4807-867f-48bad0847b73" path="/var/lib/kubelet/pods/2000e193-dc40-4807-867f-48bad0847b73/volumes"
Oct 30 13:02:25.560887 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:25.560871    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28e5ff40-671c-422e-9875-82ddc3c878ce" path="/var/lib/kubelet/pods/28e5ff40-671c-422e-9875-82ddc3c878ce/volumes"
Oct 30 13:02:25.561249 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:25.561236    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="efb9145f-43ea-4eb9-9700-cd84fbfa8459" path="/var/lib/kubelet/pods/efb9145f-43ea-4eb9-9700-cd84fbfa8459/volumes"
Oct 30 13:02:25.625374 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:25.625331    4945 patch_prober.go:29] interesting pod/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 13:02:25.625539 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:25.625398    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr" podUID="c73e1c53-dd91-4247-8f72-390a6708190d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 13:02:26.239304 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:02:26.239197    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:02:26.624556 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:26.624461    4945 patch_prober.go:29] interesting pod/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 13:02:26.624556 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:26.624524    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr" podUID="c73e1c53-dd91-4247-8f72-390a6708190d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 13:02:27.624427 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:27.624370    4945 patch_prober.go:29] interesting pod/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 13:02:27.624870 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:27.624443    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr" podUID="c73e1c53-dd91-4247-8f72-390a6708190d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 13:02:28.625165 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:28.625132    4945 patch_prober.go:29] interesting pod/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 13:02:28.625684 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:28.625209    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr" podUID="c73e1c53-dd91-4247-8f72-390a6708190d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 13:02:29.629485 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:29.629438    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:29.629967 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:29.629915    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:29.630626 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:29.630581    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:31.198796 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:31.198747    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"]
Oct 30 13:02:31.283141 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:31.283105    4945 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr" secret="" err="secret \"e2e-ipp-b-096cc3-openshift-default-dockercfg-5rpdm\" not found"
Oct 30 13:02:32.289988 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:32.289947    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr" podUID="c73e1c53-dd91-4247-8f72-390a6708190d" containerName="istio-proxy" containerID="cri-o://49671fad8fbfc6059e731d437c5a94b9a3879c9b7f18a03ed49acba5edbaab2a" gracePeriod=30
Oct 30 13:02:37.651996 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.651961    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:37.714209 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.714151    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"]
Oct 30 13:02:37.715138 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.715111    4945 cpu_manager.go:414] "containerMap: removing container" podUID="28e5ff40-671c-422e-9875-82ddc3c878ce" containerName="maas-api"
Oct 30 13:02:37.715138 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.715136    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28e5ff40-671c-422e-9875-82ddc3c878ce" containerName="maas-api"
Oct 30 13:02:37.715333 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.715166    4945 cpu_manager.go:414] "containerMap: removing container" podUID="2000e193-dc40-4807-867f-48bad0847b73" containerName="maas-api"
Oct 30 13:02:37.715333 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.715173    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2000e193-dc40-4807-867f-48bad0847b73" containerName="maas-api"
Oct 30 13:02:37.715333 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.715194    4945 cpu_manager.go:414] "containerMap: removing container" podUID="efb9145f-43ea-4eb9-9700-cd84fbfa8459" containerName="payload-pre-processing"
Oct 30 13:02:37.715333 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.715202    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efb9145f-43ea-4eb9-9700-cd84fbfa8459" containerName="payload-pre-processing"
Oct 30 13:02:37.715333 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.715227    4945 cpu_manager.go:414] "containerMap: removing container" podUID="c73e1c53-dd91-4247-8f72-390a6708190d" containerName="istio-proxy"
Oct 30 13:02:37.715333 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.715234    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c73e1c53-dd91-4247-8f72-390a6708190d" containerName="istio-proxy"
Oct 30 13:02:37.715677 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.715367    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="efb9145f-43ea-4eb9-9700-cd84fbfa8459" containerName="payload-pre-processing"
Oct 30 13:02:37.715677 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.715379    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="c73e1c53-dd91-4247-8f72-390a6708190d" containerName="istio-proxy"
Oct 30 13:02:37.715677 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.715397    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="28e5ff40-671c-422e-9875-82ddc3c878ce" containerName="maas-api"
Oct 30 13:02:37.715677 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.715407    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="2000e193-dc40-4807-867f-48bad0847b73" containerName="maas-api"
Oct 30 13:02:37.724645 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.724499    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:37.739092 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.739064    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"]
Oct 30 13:02:37.741266 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.741212    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-2382a9-serving-cert\""
Oct 30 13:02:37.751265 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.751223    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c73e1c53-dd91-4247-8f72-390a6708190d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c73e1c53-dd91-4247-8f72-390a6708190d-istiod-ca-cert\") pod \"c73e1c53-dd91-4247-8f72-390a6708190d\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") "
Oct 30 13:02:37.751565 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.751550    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-istio-envoy\") pod \"c73e1c53-dd91-4247-8f72-390a6708190d\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") "
Oct 30 13:02:37.752103 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.752082    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c73e1c53-dd91-4247-8f72-390a6708190d-istio-token\" (UniqueName: \"kubernetes.io/projected/c73e1c53-dd91-4247-8f72-390a6708190d-istio-token\") pod \"c73e1c53-dd91-4247-8f72-390a6708190d\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") "
Oct 30 13:02:37.752201 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.752140    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-credential-socket\") pod \"c73e1c53-dd91-4247-8f72-390a6708190d\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") "
Oct 30 13:02:37.752267 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.752254    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-workload-certs\") pod \"c73e1c53-dd91-4247-8f72-390a6708190d\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") "
Oct 30 13:02:37.752322 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.752307    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c73e1c53-dd91-4247-8f72-390a6708190d-kube-api-access-8fbrm\" (UniqueName: \"kubernetes.io/projected/c73e1c53-dd91-4247-8f72-390a6708190d-kube-api-access-8fbrm\") pod \"c73e1c53-dd91-4247-8f72-390a6708190d\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") "
Oct 30 13:02:37.752418 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.752402    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c73e1c53-dd91-4247-8f72-390a6708190d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c73e1c53-dd91-4247-8f72-390a6708190d-istio-podinfo\") pod \"c73e1c53-dd91-4247-8f72-390a6708190d\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") "
Oct 30 13:02:37.754983 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.751863    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c73e1c53-dd91-4247-8f72-390a6708190d-istiod-ca-cert" pod "c73e1c53-dd91-4247-8f72-390a6708190d" (UID: "c73e1c53-dd91-4247-8f72-390a6708190d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:02:37.755379 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.752982    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-workload-certs" pod "c73e1c53-dd91-4247-8f72-390a6708190d" (UID: "c73e1c53-dd91-4247-8f72-390a6708190d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:02:37.755379 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.753199    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-credential-socket" pod "c73e1c53-dd91-4247-8f72-390a6708190d" (UID: "c73e1c53-dd91-4247-8f72-390a6708190d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:02:37.755379 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.754318    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-istio-envoy" pod "c73e1c53-dd91-4247-8f72-390a6708190d" (UID: "c73e1c53-dd91-4247-8f72-390a6708190d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:02:37.755379 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.755287    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-istio-data" pod "c73e1c53-dd91-4247-8f72-390a6708190d" (UID: "c73e1c53-dd91-4247-8f72-390a6708190d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:02:37.755955 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.755921    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-istio-data\") pod \"c73e1c53-dd91-4247-8f72-390a6708190d\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") "
Oct 30 13:02:37.756133 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.756095    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-workload-socket\") pod \"c73e1c53-dd91-4247-8f72-390a6708190d\" (UID: \"c73e1c53-dd91-4247-8f72-390a6708190d\") "
Oct 30 13:02:37.756376 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.756350    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-workload-socket" pod "c73e1c53-dd91-4247-8f72-390a6708190d" (UID: "c73e1c53-dd91-4247-8f72-390a6708190d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:02:37.758470 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.758446    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2qdn\" (UniqueName: \"kubernetes.io/projected/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-kube-api-access-v2qdn\") pod \"maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:37.759730 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.759710    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-maas-api-tls\") pod \"maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:37.759909 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.759893    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-metrics-certs\") pod \"maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:37.760257 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.760238    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:37.760490 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.760477    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:37.760582 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.760572    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:37.760693 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.760681    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c73e1c53-dd91-4247-8f72-390a6708190d-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:37.760789 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.760777    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:37.760888 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.760877    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:37.760967 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.760954    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c73e1c53-dd91-4247-8f72-390a6708190d-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:37.761825 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.761794    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c73e1c53-dd91-4247-8f72-390a6708190d-istio-podinfo" pod "c73e1c53-dd91-4247-8f72-390a6708190d" (UID: "c73e1c53-dd91-4247-8f72-390a6708190d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:02:37.769637 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.767216    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c73e1c53-dd91-4247-8f72-390a6708190d-kube-api-access-8fbrm" pod "c73e1c53-dd91-4247-8f72-390a6708190d" (UID: "c73e1c53-dd91-4247-8f72-390a6708190d"). InnerVolumeSpecName "kube-api-access-8fbrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:02:37.773086 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.770180    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c73e1c53-dd91-4247-8f72-390a6708190d-istio-token" pod "c73e1c53-dd91-4247-8f72-390a6708190d" (UID: "c73e1c53-dd91-4247-8f72-390a6708190d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:02:37.862489 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.862397    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2qdn\" (UniqueName: \"kubernetes.io/projected/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-kube-api-access-v2qdn\") pod \"maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:37.862489 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.862461    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-maas-api-tls\") pod \"maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:37.862760 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.862490    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-metrics-certs\") pod \"maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:37.862760 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.862697    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:37.862872 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.862820    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fbrm\" (UniqueName: \"kubernetes.io/projected/c73e1c53-dd91-4247-8f72-390a6708190d-kube-api-access-8fbrm\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:37.862872 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.862841    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c73e1c53-dd91-4247-8f72-390a6708190d-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:37.862872 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.862856    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c73e1c53-dd91-4247-8f72-390a6708190d-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:02:37.863492 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.863464    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:37.865663 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.865642    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-metrics-certs\") pod \"maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:37.865782 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.865761    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-maas-api-tls\") pod \"maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:37.884556 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:37.884529    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2qdn\" (UniqueName: \"kubernetes.io/projected/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-kube-api-access-v2qdn\") pod \"maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:38.050238 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:38.050198    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:38.347343 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:38.347305    4945 generic.go:356] "Generic (PLEG): container finished" podID="c73e1c53-dd91-4247-8f72-390a6708190d" containerID="49671fad8fbfc6059e731d437c5a94b9a3879c9b7f18a03ed49acba5edbaab2a" exitCode=0
Oct 30 13:02:38.347489 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:38.347354    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"
Oct 30 13:02:38.347489 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:38.347366    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr" event={"ID":"c73e1c53-dd91-4247-8f72-390a6708190d","Type":"ContainerDied","Data":"49671fad8fbfc6059e731d437c5a94b9a3879c9b7f18a03ed49acba5edbaab2a"}
Oct 30 13:02:38.347489 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:38.347425    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr" event={"ID":"c73e1c53-dd91-4247-8f72-390a6708190d","Type":"ContainerDied","Data":"44123cb74e348946885c3169f2b070a1baac6cebc6754e7b0b7c094c647dfc53"}
Oct 30 13:02:38.347489 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:38.347457    4945 scope.go:122] "RemoveContainer" containerID="49671fad8fbfc6059e731d437c5a94b9a3879c9b7f18a03ed49acba5edbaab2a"
Oct 30 13:02:38.359961 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:38.359905    4945 scope.go:122] "RemoveContainer" containerID="49671fad8fbfc6059e731d437c5a94b9a3879c9b7f18a03ed49acba5edbaab2a"
Oct 30 13:02:38.360245 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:02:38.360218    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49671fad8fbfc6059e731d437c5a94b9a3879c9b7f18a03ed49acba5edbaab2a\": container with ID starting with 49671fad8fbfc6059e731d437c5a94b9a3879c9b7f18a03ed49acba5edbaab2a not found: ID does not exist" containerID="49671fad8fbfc6059e731d437c5a94b9a3879c9b7f18a03ed49acba5edbaab2a"
Oct 30 13:02:38.360343 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:38.360258    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49671fad8fbfc6059e731d437c5a94b9a3879c9b7f18a03ed49acba5edbaab2a"} err="failed to get container status \"49671fad8fbfc6059e731d437c5a94b9a3879c9b7f18a03ed49acba5edbaab2a\": rpc error: code = NotFound desc = could not find container \"49671fad8fbfc6059e731d437c5a94b9a3879c9b7f18a03ed49acba5edbaab2a\": container with ID starting with 49671fad8fbfc6059e731d437c5a94b9a3879c9b7f18a03ed49acba5edbaab2a not found: ID does not exist"
Oct 30 13:02:38.362737 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:38.362714    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"]
Oct 30 13:02:38.364738 ip-10-0-142-36 kubenswrapper[4945]: W1030 13:02:38.364711    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod368bc543_c9e7_4fd6_a3b3_a147af5b4be3.slice/crio-93b3466756bdcd5fd0d3320515b54eeb34b56a9bcd1e5eb9e8b470d5e3a1cbc2 WatchSource:0}: Error finding container 93b3466756bdcd5fd0d3320515b54eeb34b56a9bcd1e5eb9e8b470d5e3a1cbc2: Status 404 returned error can't find the container with id 93b3466756bdcd5fd0d3320515b54eeb34b56a9bcd1e5eb9e8b470d5e3a1cbc2
Oct 30 13:02:38.401970 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:38.401880    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"]
Oct 30 13:02:38.405489 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:38.405404    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-096cc3-openshift-default-5c98c69cb5-p6pwr"]
Oct 30 13:02:39.359293 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:39.359228    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52" event={"ID":"368bc543-c9e7-4fd6-a3b3-a147af5b4be3","Type":"ContainerStarted","Data":"5d60d257ad5caf7ea20300df1e6f89976b46ae92e3905cfcbbc91ccd4470042b"}
Oct 30 13:02:39.359293 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:39.359304    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52" event={"ID":"368bc543-c9e7-4fd6-a3b3-a147af5b4be3","Type":"ContainerStarted","Data":"93b3466756bdcd5fd0d3320515b54eeb34b56a9bcd1e5eb9e8b470d5e3a1cbc2"}
Oct 30 13:02:39.360308 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:39.360284    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:39.402349 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:39.402290    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52" podStartSLOduration=2.066746187 podStartE2EDuration="2.402276106s" totalImagesPullingTime="335.529919ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:02:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:02:39.397512398 +0000 UTC m=+1603.975461487" watchObservedRunningTime="2026-10-30 13:02:39.402276106 +0000 UTC m=+1603.980225194"
Oct 30 13:02:39.551449 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:39.551413    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c73e1c53-dd91-4247-8f72-390a6708190d" path="/var/lib/kubelet/pods/c73e1c53-dd91-4247-8f72-390a6708190d/volumes"
Oct 30 13:02:46.398872 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:46.398841    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:02:56.197019 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:02:56.196977    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:02:56.657179 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.657104    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"]
Oct 30 13:02:56.664554 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.664533    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:02:56.681396 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.681367    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"]
Oct 30 13:02:56.687460 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.687439    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-ba9a8c-serving-cert\""
Oct 30 13:02:56.797162 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.797131    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73a5762d-46ba-4f8d-a9c4-89c948315228-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:02:56.797343 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.797186    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73a5762d-46ba-4f8d-a9c4-89c948315228-metrics-certs\") pod \"maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:02:56.797343 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.797312    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73a5762d-46ba-4f8d-a9c4-89c948315228-maas-api-tls\") pod \"maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:02:56.797441 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.797369    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9cwc\" (UniqueName: \"kubernetes.io/projected/73a5762d-46ba-4f8d-a9c4-89c948315228-kube-api-access-z9cwc\") pod \"maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:02:56.906626 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.901695    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73a5762d-46ba-4f8d-a9c4-89c948315228-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:02:56.906626 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.901819    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73a5762d-46ba-4f8d-a9c4-89c948315228-metrics-certs\") pod \"maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:02:56.906626 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.901909    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73a5762d-46ba-4f8d-a9c4-89c948315228-maas-api-tls\") pod \"maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:02:56.906626 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.902050    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9cwc\" (UniqueName: \"kubernetes.io/projected/73a5762d-46ba-4f8d-a9c4-89c948315228-kube-api-access-z9cwc\") pod \"maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:02:56.921576 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.921471    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73a5762d-46ba-4f8d-a9c4-89c948315228-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:02:56.927958 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.927861    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9cwc\" (UniqueName: \"kubernetes.io/projected/73a5762d-46ba-4f8d-a9c4-89c948315228-kube-api-access-z9cwc\") pod \"maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:02:56.929866 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.928996    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73a5762d-46ba-4f8d-a9c4-89c948315228-metrics-certs\") pod \"maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:02:56.929866 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.929814    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73a5762d-46ba-4f8d-a9c4-89c948315228-maas-api-tls\") pod \"maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:02:56.991101 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:56.991066    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:02:57.309754 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:57.309465    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"]
Oct 30 13:02:57.583676 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:57.579717    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk" event={"ID":"73a5762d-46ba-4f8d-a9c4-89c948315228","Type":"ContainerStarted","Data":"2daeeb125ae9d5b5a7fa00438a20a07f9ba3bf2bffaae5494b2fefed6eedcf42"}
Oct 30 13:02:58.592744 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:58.592702    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk" event={"ID":"73a5762d-46ba-4f8d-a9c4-89c948315228","Type":"ContainerStarted","Data":"f00d4017ee95fd41eafddb89d0574efc7328c53d6d05ae4daa47f3d8401285e0"}
Oct 30 13:02:58.593273 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:02:58.592803    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:03:04.607145 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:04.607116    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:03:04.655671 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:04.655594    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk" podStartSLOduration=8.216189629 podStartE2EDuration="8.655575485s" totalImagesPullingTime="439.385856ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:02:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:02:58.635892141 +0000 UTC m=+1623.213841244" watchObservedRunningTime="2026-10-30 13:03:04.655575485 +0000 UTC m=+1629.233524575"
Oct 30 13:03:20.407624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:20.406698    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:23.161625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:23.160754    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:23.186623 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:23.178043    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:23.200954 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:23.200907    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 13:03:23.276816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:23.276752    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:23.537247 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:23.536231    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:23.537247 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:23.536666    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 13:03:24.019587 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:24.019545    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:24.178990 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:24.178741    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 13:03:24.179491 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:24.179167    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:24.193659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:24.179776    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 13:03:24.193659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:24.179842    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 13:03:24.233646 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:24.213574    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:24.282683 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:24.272933    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 13:03:24.282683 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:24.281813    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:24.282683 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:24.282458    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 13:03:24.584684 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:24.577083    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:03:24.588734 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:24.586240    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:24.628398 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:24.586291    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 13:03:24.628398 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:24.597919    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 13:03:24.628398 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:24.598007    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 13:03:25.184781 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:25.184726    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:25.229624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:25.203984    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 13:03:25.330720 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:25.330674    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 13:03:25.394658 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:25.379712    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:25.618506 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:25.603871    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 13:03:25.618506 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:25.604153    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:25.630726 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:25.630682    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:25.717625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:25.716747    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:03:25.717625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:25.716825    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:25.744624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:25.718438    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 13:03:25.792623 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:25.792090    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:03:25.792623 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:25.792162    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw" podUID="b87eaba7-33de-46f3-a25d-2e86f1e12abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:26.103624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:26.102695    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:03:26.103624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:26.102760    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" podUID="473f256f-7ee3-4e49-b407-346e0242fcbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:26.210622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:26.209717    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:26.254648 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:26.233725    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:26.324126 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:26.322137    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 13:03:26.337622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:26.335697    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:26.399170 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:03:26.399080    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:03:26.411254 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:26.410698    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:26.550383 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:26.550353    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 13:03:26.702684 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:26.700285    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:26.702684 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:26.700840    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:26.814956 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:26.814916    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 13:03:27.252729 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:27.226202    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:27.281743 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:27.281694    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:27.355330 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:27.354076    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 13:03:27.412453 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:27.412408    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:27.510473 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:27.510388    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 13:03:27.672691 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:27.672639    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:03:27.673069 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:27.673036    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 13:03:27.769289 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:27.769196    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 13:03:27.773085 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:27.773058    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 13:03:41.252395 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:41.252361    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"]
Oct 30 13:03:41.252855 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:41.252608    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz" podUID="2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7" containerName="maas-api" containerID="cri-o://390d6575c7e80eb7ce57bf4394e5ea66b0861c1f67469c4050f235b7dfa515cf" gracePeriod=30
Oct 30 13:03:41.848804 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:41.848770    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:03:41.925333 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:41.925295    4945 generic.go:356] "Generic (PLEG): container finished" podID="2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7" containerID="390d6575c7e80eb7ce57bf4394e5ea66b0861c1f67469c4050f235b7dfa515cf" exitCode=0
Oct 30 13:03:41.925525 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:41.925502    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz" event={"ID":"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7","Type":"ContainerDied","Data":"390d6575c7e80eb7ce57bf4394e5ea66b0861c1f67469c4050f235b7dfa515cf"}
Oct 30 13:03:41.925584 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:41.925556    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz" event={"ID":"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7","Type":"ContainerDied","Data":"7d4d785b2c6e5ed391af3d52d2f3b134e18211764dadf8d518efbdb6027ac47a"}
Oct 30 13:03:41.925651 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:41.925587    4945 scope.go:122] "RemoveContainer" containerID="390d6575c7e80eb7ce57bf4394e5ea66b0861c1f67469c4050f235b7dfa515cf"
Oct 30 13:03:41.926453 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:41.926415    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"
Oct 30 13:03:41.955825 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:41.947486    4945 scope.go:122] "RemoveContainer" containerID="390d6575c7e80eb7ce57bf4394e5ea66b0861c1f67469c4050f235b7dfa515cf"
Oct 30 13:03:41.955825 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:03:41.947855    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"390d6575c7e80eb7ce57bf4394e5ea66b0861c1f67469c4050f235b7dfa515cf\": container with ID starting with 390d6575c7e80eb7ce57bf4394e5ea66b0861c1f67469c4050f235b7dfa515cf not found: ID does not exist" containerID="390d6575c7e80eb7ce57bf4394e5ea66b0861c1f67469c4050f235b7dfa515cf"
Oct 30 13:03:41.955825 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:41.947895    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390d6575c7e80eb7ce57bf4394e5ea66b0861c1f67469c4050f235b7dfa515cf"} err="failed to get container status \"390d6575c7e80eb7ce57bf4394e5ea66b0861c1f67469c4050f235b7dfa515cf\": rpc error: code = NotFound desc = could not find container \"390d6575c7e80eb7ce57bf4394e5ea66b0861c1f67469c4050f235b7dfa515cf\": container with ID starting with 390d6575c7e80eb7ce57bf4394e5ea66b0861c1f67469c4050f235b7dfa515cf not found: ID does not exist"
Oct 30 13:03:42.016622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:42.015976    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-metrics-certs\") pod \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") "
Oct 30 13:03:42.016622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:42.016272    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-kube-api-access-jtwfd\" (UniqueName: \"kubernetes.io/projected/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-kube-api-access-jtwfd\") pod \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") "
Oct 30 13:03:42.016622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:42.016485    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-openshift-service-ca\") pod \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") "
Oct 30 13:03:42.016622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:42.016577    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-maas-api-tls\") pod \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\" (UID: \"2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7\") "
Oct 30 13:03:42.019942 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:42.017981    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-openshift-service-ca" pod "2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7" (UID: "2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:03:42.019942 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:42.019223    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-kube-api-access-jtwfd" pod "2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7" (UID: "2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7"). InnerVolumeSpecName "kube-api-access-jtwfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:03:42.024721 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:42.023047    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-maas-api-tls" pod "2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7" (UID: "2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:03:42.025979 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:42.025808    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-metrics-certs" pod "2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7" (UID: "2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:03:42.121573 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:42.121531    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:03:42.121573 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:42.121565    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:03:42.121573 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:42.121580    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:03:42.121894 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:42.121592    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtwfd\" (UniqueName: \"kubernetes.io/projected/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7-kube-api-access-jtwfd\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:03:42.295292 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:42.295257    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"]
Oct 30 13:03:42.309132 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:42.309099    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-87f1b5-5d55476858-j96nz"]
Oct 30 13:03:43.552085 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:03:43.552041    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7" path="/var/lib/kubelet/pods/2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7/volumes"
Oct 30 13:03:56.193954 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:03:56.193906    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:04:00.826023 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.825978    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"]
Oct 30 13:04:00.826688 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.826671    4945 cpu_manager.go:414] "containerMap: removing container" podUID="2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7" containerName="maas-api"
Oct 30 13:04:00.826737 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.826688    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7" containerName="maas-api"
Oct 30 13:04:00.826816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.826807    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="2fcf4e3a-dfbd-4ceb-bd7f-2e98fd2ba0b7" containerName="maas-api"
Oct 30 13:04:00.831785 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.831766    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:00.851573 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.851546    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"]
Oct 30 13:04:00.857554 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.857451    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-1f91c1-serving-cert\""
Oct 30 13:04:00.864088 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.864059    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96z6s\" (UniqueName: \"kubernetes.io/projected/81e50924-941f-4d0e-a9a3-c7264a448254-kube-api-access-96z6s\") pod \"maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:00.864219 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.864191    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81e50924-941f-4d0e-a9a3-c7264a448254-metrics-certs\") pod \"maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:00.864386 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.864361    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81e50924-941f-4d0e-a9a3-c7264a448254-maas-api-tls\") pod \"maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:00.864443 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.864409    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e50924-941f-4d0e-a9a3-c7264a448254-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:00.965869 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.965835    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81e50924-941f-4d0e-a9a3-c7264a448254-maas-api-tls\") pod \"maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:00.965869 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.965876    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e50924-941f-4d0e-a9a3-c7264a448254-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:00.966110 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.966049    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96z6s\" (UniqueName: \"kubernetes.io/projected/81e50924-941f-4d0e-a9a3-c7264a448254-kube-api-access-96z6s\") pod \"maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:00.966174 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.966138    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81e50924-941f-4d0e-a9a3-c7264a448254-metrics-certs\") pod \"maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:00.966729 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.966702    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e50924-941f-4d0e-a9a3-c7264a448254-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:00.968504 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.968481    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81e50924-941f-4d0e-a9a3-c7264a448254-metrics-certs\") pod \"maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:00.968641 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.968543    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81e50924-941f-4d0e-a9a3-c7264a448254-maas-api-tls\") pod \"maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:00.986133 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:00.986106    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96z6s\" (UniqueName: \"kubernetes.io/projected/81e50924-941f-4d0e-a9a3-c7264a448254-kube-api-access-96z6s\") pod \"maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:01.146032 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:01.145941    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:01.474256 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:01.474224    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"]
Oct 30 13:04:01.475711 ip-10-0-142-36 kubenswrapper[4945]: W1030 13:04:01.475683    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81e50924_941f_4d0e_a9a3_c7264a448254.slice/crio-7af5b9fe20849ece7b9838b7651ddbd48812836b66cffd342bb3dba52f72d626 WatchSource:0}: Error finding container 7af5b9fe20849ece7b9838b7651ddbd48812836b66cffd342bb3dba52f72d626: Status 404 returned error can't find the container with id 7af5b9fe20849ece7b9838b7651ddbd48812836b66cffd342bb3dba52f72d626
Oct 30 13:04:02.144199 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:02.144154    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958" event={"ID":"81e50924-941f-4d0e-a9a3-c7264a448254","Type":"ContainerStarted","Data":"7af5b9fe20849ece7b9838b7651ddbd48812836b66cffd342bb3dba52f72d626"}
Oct 30 13:04:03.152154 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:03.152118    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958" event={"ID":"81e50924-941f-4d0e-a9a3-c7264a448254","Type":"ContainerStarted","Data":"2312bf9c056236a1f84c36270900194d7f65fb88b8b34f9402b12387c8aec205"}
Oct 30 13:04:03.152546 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:03.152317    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:03.195265 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:03.195211    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958" podStartSLOduration=2.692002423 podStartE2EDuration="3.195197403s" totalImagesPullingTime="503.19498ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:04:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:04:03.191629325 +0000 UTC m=+1687.769578408" watchObservedRunningTime="2026-10-30 13:04:03.195197403 +0000 UTC m=+1687.773146537"
Oct 30 13:04:06.270554 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.270520    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"]
Oct 30 13:04:06.279044 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.278995    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:06.298634 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.298225    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"]
Oct 30 13:04:06.323332 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.323286    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-8f3cb8-serving-cert\""
Oct 30 13:04:06.354503 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.354465    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bae96c60-5094-420b-8dd6-8bba39c588a8-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:06.354707 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.354533    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czv4h\" (UniqueName: \"kubernetes.io/projected/bae96c60-5094-420b-8dd6-8bba39c588a8-kube-api-access-czv4h\") pod \"maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:06.354707 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.354639    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-metrics-certs\") pod \"maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:06.354707 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.354681    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-maas-api-tls\") pod \"maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:06.455793 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.455761    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bae96c60-5094-420b-8dd6-8bba39c588a8-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:06.455996 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.455804    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czv4h\" (UniqueName: \"kubernetes.io/projected/bae96c60-5094-420b-8dd6-8bba39c588a8-kube-api-access-czv4h\") pod \"maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:06.455996 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.455824    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-metrics-certs\") pod \"maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:06.455996 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.455847    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-maas-api-tls\") pod \"maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:06.456167 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:04:06.456027    4945 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-serving-cert: secret "maas-api-e2e-praxis-b-8f3cb8-serving-cert" not found
Oct 30 13:04:06.456167 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:04:06.456097    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-maas-api-tls podName:bae96c60-5094-420b-8dd6-8bba39c588a8 nodeName:}" failed. No retries permitted until 2026-10-30 13:04:06.956078339 +0000 UTC m=+1691.534027437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-maas-api-tls") pod "maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr" (UID: "bae96c60-5094-420b-8dd6-8bba39c588a8") : secret "maas-api-e2e-praxis-b-8f3cb8-serving-cert" not found
Oct 30 13:04:06.456742 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.456716    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bae96c60-5094-420b-8dd6-8bba39c588a8-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:06.458423 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.458400    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-metrics-certs\") pod \"maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:06.505299 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.505237    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czv4h\" (UniqueName: \"kubernetes.io/projected/bae96c60-5094-420b-8dd6-8bba39c588a8-kube-api-access-czv4h\") pod \"maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:06.963269 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.963228    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-maas-api-tls\") pod \"maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:06.965839 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:06.965818    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-maas-api-tls\") pod \"maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:07.226487 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:07.226358    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:07.620150 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:07.620123    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"]
Oct 30 13:04:08.204836 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:08.204794    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr" event={"ID":"bae96c60-5094-420b-8dd6-8bba39c588a8","Type":"ContainerStarted","Data":"9f0da851d16460dc5b2355d38975167d04ce2ae6fc6a9cfc51dfa90652ff392a"}
Oct 30 13:04:08.205034 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:08.204860    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr" event={"ID":"bae96c60-5094-420b-8dd6-8bba39c588a8","Type":"ContainerStarted","Data":"64870ce8ecc261a40eb3ef55af3e63d8ed57b6c9e5982ecd5fc31e29a3191272"}
Oct 30 13:04:08.205034 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:08.204890    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:08.248375 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:08.248282    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr" podStartSLOduration=1.9220586160000002 podStartE2EDuration="2.24826785s" totalImagesPullingTime="326.209234ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:04:08.243896666 +0000 UTC m=+1692.821845756" watchObservedRunningTime="2026-10-30 13:04:08.24826785 +0000 UTC m=+1692.826216938"
Oct 30 13:04:09.165593 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:09.165566    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:04:14.221077 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:14.221046    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:04:18.369837 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:18.369804    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"]
Oct 30 13:04:18.374102 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:18.373832    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:18.393076 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:18.393047    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-f73c9f-serving-cert\""
Oct 30 13:04:18.411637 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:18.411557    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"]
Oct 30 13:04:18.444418 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:18.444385    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:18.444628 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:18.444577    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:18.444802 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:18.444779    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5fhz\" (UniqueName: \"kubernetes.io/projected/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-kube-api-access-s5fhz\") pod \"maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:18.444860 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:18.444843    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-metrics-certs\") pod \"maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:18.546461 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:18.546428    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:18.546705 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:18.546550    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:18.546705 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:18.546668    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5fhz\" (UniqueName: \"kubernetes.io/projected/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-kube-api-access-s5fhz\") pod \"maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:18.546852 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:18.546710    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-metrics-certs\") pod \"maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:18.546852 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:04:18.546719    4945 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-serving-cert: secret "maas-api-e2e-praxis-a-f73c9f-serving-cert" not found
Oct 30 13:04:18.546852 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:04:18.546799    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-maas-api-tls podName:55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e nodeName:}" failed. No retries permitted until 2026-10-30 13:04:19.046776766 +0000 UTC m=+1703.624725856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-maas-api-tls") pod "maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td" (UID: "55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e") : secret "maas-api-e2e-praxis-a-f73c9f-serving-cert" not found
Oct 30 13:04:18.547205 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:18.547181    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:18.549282 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:18.549255    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-metrics-certs\") pod \"maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:18.577644 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:18.573336    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5fhz\" (UniqueName: \"kubernetes.io/projected/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-kube-api-access-s5fhz\") pod \"maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:19.054140 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:19.054104    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:19.056539 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:19.056512    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-maas-api-tls\") pod \"maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:19.289257 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:19.289224    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:19.689088 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:19.687524    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"]
Oct 30 13:04:19.718829 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:19.699947    4945 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 13:04:20.084461 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.084428    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"]
Oct 30 13:04:20.085364 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.085311    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52" podUID="368bc543-c9e7-4fd6-a3b3-a147af5b4be3" containerName="maas-api" containerID="cri-o://5d60d257ad5caf7ea20300df1e6f89976b46ae92e3905cfcbbc91ccd4470042b" gracePeriod=30
Oct 30 13:04:20.303142 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.302051    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td" event={"ID":"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e","Type":"ContainerStarted","Data":"591e2b77d5762913353bf21fb6e26e5a7b1026d189fc7f678b62aabe1f1a4099"}
Oct 30 13:04:20.303142 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.302116    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td" event={"ID":"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e","Type":"ContainerStarted","Data":"65ea8a6783ed5387a5e47cc1b0982cf3fa3e64b6834c6805734c6bd1bea62996"}
Oct 30 13:04:20.303142 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.303058    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:20.305503 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.305321    4945 generic.go:356] "Generic (PLEG): container finished" podID="368bc543-c9e7-4fd6-a3b3-a147af5b4be3" containerID="5d60d257ad5caf7ea20300df1e6f89976b46ae92e3905cfcbbc91ccd4470042b" exitCode=0
Oct 30 13:04:20.305503 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.305449    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52" event={"ID":"368bc543-c9e7-4fd6-a3b3-a147af5b4be3","Type":"ContainerDied","Data":"5d60d257ad5caf7ea20300df1e6f89976b46ae92e3905cfcbbc91ccd4470042b"}
Oct 30 13:04:20.353115 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.346185    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td" podStartSLOduration=2.005944456 podStartE2EDuration="2.34615435s" totalImagesPullingTime="340.209894ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:04:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:04:20.344821965 +0000 UTC m=+1704.922771054" watchObservedRunningTime="2026-10-30 13:04:20.34615435 +0000 UTC m=+1704.924103443"
Oct 30 13:04:20.445151 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.445120    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:04:20.499651 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.499616    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-kube-api-access-v2qdn\" (UniqueName: \"kubernetes.io/projected/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-kube-api-access-v2qdn\") pod \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") "
Oct 30 13:04:20.499833 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.499749    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-maas-api-tls\") pod \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") "
Oct 30 13:04:20.499833 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.499817    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-openshift-service-ca\") pod \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") "
Oct 30 13:04:20.501542 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.501512    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-openshift-service-ca" pod "368bc543-c9e7-4fd6-a3b3-a147af5b4be3" (UID: "368bc543-c9e7-4fd6-a3b3-a147af5b4be3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:04:20.504801 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.504765    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-kube-api-access-v2qdn" pod "368bc543-c9e7-4fd6-a3b3-a147af5b4be3" (UID: "368bc543-c9e7-4fd6-a3b3-a147af5b4be3"). InnerVolumeSpecName "kube-api-access-v2qdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:04:20.505744 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.505723    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-maas-api-tls" pod "368bc543-c9e7-4fd6-a3b3-a147af5b4be3" (UID: "368bc543-c9e7-4fd6-a3b3-a147af5b4be3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:04:20.602101 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.602059    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-metrics-certs\") pod \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\" (UID: \"368bc543-c9e7-4fd6-a3b3-a147af5b4be3\") "
Oct 30 13:04:20.603912 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.603094    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:04:20.603912 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.603121    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:04:20.603912 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.603137    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2qdn\" (UniqueName: \"kubernetes.io/projected/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-kube-api-access-v2qdn\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:04:20.604871 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.604833    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-metrics-certs" pod "368bc543-c9e7-4fd6-a3b3-a147af5b4be3" (UID: "368bc543-c9e7-4fd6-a3b3-a147af5b4be3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:04:20.707781 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:20.707749    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/368bc543-c9e7-4fd6-a3b3-a147af5b4be3-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:04:21.318839 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:21.318808    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"
Oct 30 13:04:21.318839 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:21.318822    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52" event={"ID":"368bc543-c9e7-4fd6-a3b3-a147af5b4be3","Type":"ContainerDied","Data":"93b3466756bdcd5fd0d3320515b54eeb34b56a9bcd1e5eb9e8b470d5e3a1cbc2"}
Oct 30 13:04:21.319087 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:21.318888    4945 scope.go:122] "RemoveContainer" containerID="5d60d257ad5caf7ea20300df1e6f89976b46ae92e3905cfcbbc91ccd4470042b"
Oct 30 13:04:21.380199 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:21.380170    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"]
Oct 30 13:04:21.393858 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:21.393832    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-2382a9-869cfc69f5-vnd52"]
Oct 30 13:04:21.549885 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:21.549851    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="368bc543-c9e7-4fd6-a3b3-a147af5b4be3" path="/var/lib/kubelet/pods/368bc543-c9e7-4fd6-a3b3-a147af5b4be3/volumes"
Oct 30 13:04:26.192469 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:04:26.192428    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:04:27.332497 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:27.332468    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:04:36.717623 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:36.697795    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:36.718831 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:36.718790    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:38.684625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:38.683914    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:38.703625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:38.685283    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 13:04:38.703625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:38.685659    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:04:38.703625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:38.699405    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": context deadline exceeded"
Oct 30 13:04:38.703625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:38.699800    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 13:04:38.736102 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:38.736049    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:39.687807 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:39.687583    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:04:39.885521 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:39.859703    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:40.066625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:39.975650    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:40.066625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:39.976463    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:40.216724 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:40.216690    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 13:04:40.217124 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:40.217106    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 13:04:40.217368 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:40.217317    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 13:04:40.218682 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:40.218664    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 13:04:40.474387 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:40.474332    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:40.715626 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:40.495727    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:40.715626 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:40.589870    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 13:04:40.715626 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:40.653752    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 13:04:40.715626 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:40.711760    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:04:40.801733 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:40.769765    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:04:40.801733 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:40.769848    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:40.803790 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:40.803750    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:04:40.803942 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:40.803848    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw" podUID="b87eaba7-33de-46f3-a25d-2e86f1e12abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:40.845115 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:40.845081    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 13:04:40.976974 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:40.976066    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:41.015928 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:41.015326    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 13:04:41.108837 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:41.108734    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:04:41.109114 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:41.109064    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" podUID="473f256f-7ee3-4e49-b407-346e0242fcbb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:41.674622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:41.652075    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:41.674622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:41.652556    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 13:04:41.983139 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:41.983097    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:42.556782 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:42.556737    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": context deadline exceeded"
Oct 30 13:04:42.618239 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:42.618195    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:42.654043 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:42.653684    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:42.709811 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:42.709756    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:42.867838 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:42.867731    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:42.900197 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:42.900150    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:42.987855 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:42.987808    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:43.054780 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:43.054731    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:43.298635 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:43.296683    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 13:04:43.659672 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:43.654680    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 13:04:43.668621 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:43.667710    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:43.668621 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:43.668331    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 13:04:43.723630 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:43.719683    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded"
Oct 30 13:04:43.723630 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:43.721770    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 13:04:44.040452 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:44.040405    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:04:44.041544 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:44.041419    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": context deadline exceeded"
Oct 30 13:04:44.070370 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:44.070331    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:04:44.113127 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:44.112630    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 13:04:44.127156 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:44.127128    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 13:04:44.127767 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:44.127746    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 13:04:44.152921 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:44.143902    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 13:04:44.154496 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:44.154475    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 13:04:44.156946 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:44.156926    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 13:04:56.200632 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:04:56.200579    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:04:57.438738 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:57.438701    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"]
Oct 30 13:04:57.452064 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:04:57.452028    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"]
Oct 30 13:05:12.471042 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.470923    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" containerID="cri-o://2355652ea89996073cb52d42e22aab8704aa65c36bf021a2061c5c42f882ec73" gracePeriod=45
Oct 30 13:05:12.542984 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.542918    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main" containerID="cri-o://5926a6fbae2c0783eb6a44082f9d8a6ea6077a61e88281228d98ab99a01c6765" gracePeriod=45
Oct 30 13:05:12.654299 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.654262    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": dial tcp 10.132.0.82:8000: connect: connection refused"
Oct 30 13:05:12.905231 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.904865    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 13:05:12.917389 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.917068    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/699038e0-b6a8-432d-96fb-bea730c1840a-tls-certs\" (UniqueName: \"kubernetes.io/secret/699038e0-b6a8-432d-96fb-bea730c1840a-tls-certs\") pod \"699038e0-b6a8-432d-96fb-bea730c1840a\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") "
Oct 30 13:05:12.917389 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.917167    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-tmp-dir\") pod \"699038e0-b6a8-432d-96fb-bea730c1840a\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") "
Oct 30 13:05:12.917389 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.917235    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-model-cache\") pod \"699038e0-b6a8-432d-96fb-bea730c1840a\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") "
Oct 30 13:05:12.917389 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.917349    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-home\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-home\") pod \"699038e0-b6a8-432d-96fb-bea730c1840a\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") "
Oct 30 13:05:12.917389 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.917383    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/699038e0-b6a8-432d-96fb-bea730c1840a-kube-api-access-8rn6w\" (UniqueName: \"kubernetes.io/projected/699038e0-b6a8-432d-96fb-bea730c1840a-kube-api-access-8rn6w\") pod \"699038e0-b6a8-432d-96fb-bea730c1840a\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") "
Oct 30 13:05:12.922807 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.917521    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-dshm\") pod \"699038e0-b6a8-432d-96fb-bea730c1840a\" (UID: \"699038e0-b6a8-432d-96fb-bea730c1840a\") "
Oct 30 13:05:12.922807 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.922092    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-home" pod "699038e0-b6a8-432d-96fb-bea730c1840a" (UID: "699038e0-b6a8-432d-96fb-bea730c1840a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:12.922807 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.922105    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-model-cache" pod "699038e0-b6a8-432d-96fb-bea730c1840a" (UID: "699038e0-b6a8-432d-96fb-bea730c1840a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:12.959144 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.946743    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-tmp-dir" pod "699038e0-b6a8-432d-96fb-bea730c1840a" (UID: "699038e0-b6a8-432d-96fb-bea730c1840a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:12.959144 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.954234    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/699038e0-b6a8-432d-96fb-bea730c1840a-kube-api-access-8rn6w" pod "699038e0-b6a8-432d-96fb-bea730c1840a" (UID: "699038e0-b6a8-432d-96fb-bea730c1840a"). InnerVolumeSpecName "kube-api-access-8rn6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:05:12.959144 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.957251    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/699038e0-b6a8-432d-96fb-bea730c1840a-tls-certs" pod "699038e0-b6a8-432d-96fb-bea730c1840a" (UID: "699038e0-b6a8-432d-96fb-bea730c1840a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:05:12.965481 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.962540    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-dshm" pod "699038e0-b6a8-432d-96fb-bea730c1840a" (UID: "699038e0-b6a8-432d-96fb-bea730c1840a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:12.975831 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:12.975564    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 13:05:13.023916 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.022009    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-model-cache\") pod \"fbff5ead-c33b-4cce-951d-dc130152f708\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") "
Oct 30 13:05:13.023916 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.022140    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-dshm\") pod \"fbff5ead-c33b-4cce-951d-dc130152f708\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") "
Oct 30 13:05:13.023916 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.022289    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fbff5ead-c33b-4cce-951d-dc130152f708-tls-certs\" (UniqueName: \"kubernetes.io/secret/fbff5ead-c33b-4cce-951d-dc130152f708-tls-certs\") pod \"fbff5ead-c33b-4cce-951d-dc130152f708\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") "
Oct 30 13:05:13.023916 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.022408    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-tmp-dir\") pod \"fbff5ead-c33b-4cce-951d-dc130152f708\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") "
Oct 30 13:05:13.023916 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.022513    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-home\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-home\") pod \"fbff5ead-c33b-4cce-951d-dc130152f708\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") "
Oct 30 13:05:13.023916 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.022543    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbff5ead-c33b-4cce-951d-dc130152f708-kube-api-access-7xqqm\" (UniqueName: \"kubernetes.io/projected/fbff5ead-c33b-4cce-951d-dc130152f708-kube-api-access-7xqqm\") pod \"fbff5ead-c33b-4cce-951d-dc130152f708\" (UID: \"fbff5ead-c33b-4cce-951d-dc130152f708\") "
Oct 30 13:05:13.023916 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.023269    4945 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-home\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:13.023916 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.023298    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rn6w\" (UniqueName: \"kubernetes.io/projected/699038e0-b6a8-432d-96fb-bea730c1840a-kube-api-access-8rn6w\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:13.023916 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.023311    4945 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-dshm\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:13.023916 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.023324    4945 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/699038e0-b6a8-432d-96fb-bea730c1840a-tls-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:13.023916 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.023340    4945 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-tmp-dir\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:13.023916 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.023353    4945 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/699038e0-b6a8-432d-96fb-bea730c1840a-model-cache\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:13.027383 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.027354    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-home" pod "fbff5ead-c33b-4cce-951d-dc130152f708" (UID: "fbff5ead-c33b-4cce-951d-dc130152f708"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:13.029773 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.029725    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbff5ead-c33b-4cce-951d-dc130152f708-kube-api-access-7xqqm" pod "fbff5ead-c33b-4cce-951d-dc130152f708" (UID: "fbff5ead-c33b-4cce-951d-dc130152f708"). InnerVolumeSpecName "kube-api-access-7xqqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:05:13.031588 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.031521    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-tmp-dir" pod "fbff5ead-c33b-4cce-951d-dc130152f708" (UID: "fbff5ead-c33b-4cce-951d-dc130152f708"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:13.031760 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.031623    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-model-cache" pod "fbff5ead-c33b-4cce-951d-dc130152f708" (UID: "fbff5ead-c33b-4cce-951d-dc130152f708"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:13.039578 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.034360    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-dshm" pod "fbff5ead-c33b-4cce-951d-dc130152f708" (UID: "fbff5ead-c33b-4cce-951d-dc130152f708"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:13.040285 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.040245    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbff5ead-c33b-4cce-951d-dc130152f708-tls-certs" pod "fbff5ead-c33b-4cce-951d-dc130152f708" (UID: "fbff5ead-c33b-4cce-951d-dc130152f708"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:05:13.124883 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.124850    4945 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-home\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:13.124883 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.124884    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xqqm\" (UniqueName: \"kubernetes.io/projected/fbff5ead-c33b-4cce-951d-dc130152f708-kube-api-access-7xqqm\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:13.125239 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.124900    4945 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-model-cache\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:13.125239 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.124915    4945 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-dshm\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:13.125239 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.124927    4945 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fbff5ead-c33b-4cce-951d-dc130152f708-tls-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:13.125239 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.124939    4945 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fbff5ead-c33b-4cce-951d-dc130152f708-tmp-dir\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:13.400706 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.400586    4945 generic.go:356] "Generic (PLEG): container finished" podID="fbff5ead-c33b-4cce-951d-dc130152f708" containerID="5926a6fbae2c0783eb6a44082f9d8a6ea6077a61e88281228d98ab99a01c6765" exitCode=0
Oct 30 13:05:13.400706 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.400632    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" event={"ID":"fbff5ead-c33b-4cce-951d-dc130152f708","Type":"ContainerDied","Data":"5926a6fbae2c0783eb6a44082f9d8a6ea6077a61e88281228d98ab99a01c6765"}
Oct 30 13:05:13.400706 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.400678    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"
Oct 30 13:05:13.400706 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.400694    4945 scope.go:122] "RemoveContainer" containerID="5926a6fbae2c0783eb6a44082f9d8a6ea6077a61e88281228d98ab99a01c6765"
Oct 30 13:05:13.401037 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.400682    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj" event={"ID":"fbff5ead-c33b-4cce-951d-dc130152f708","Type":"ContainerDied","Data":"7075f06fa09a028ac52628a5b92c79cc0bb09bc965bb683ccd7297cf522d5b9b"}
Oct 30 13:05:13.402728 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.402702    4945 generic.go:356] "Generic (PLEG): container finished" podID="699038e0-b6a8-432d-96fb-bea730c1840a" containerID="2355652ea89996073cb52d42e22aab8704aa65c36bf021a2061c5c42f882ec73" exitCode=0
Oct 30 13:05:13.402834 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.402741    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"
Oct 30 13:05:13.402834 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.402780    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" event={"ID":"699038e0-b6a8-432d-96fb-bea730c1840a","Type":"ContainerDied","Data":"2355652ea89996073cb52d42e22aab8704aa65c36bf021a2061c5c42f882ec73"}
Oct 30 13:05:13.402834 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.402813    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4" event={"ID":"699038e0-b6a8-432d-96fb-bea730c1840a","Type":"ContainerDied","Data":"7d4f2f83a876a1c7d28337e7f4cdb23515eb2f5a5d6c1deacc26284f7eccf04f"}
Oct 30 13:05:13.412835 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.412816    4945 scope.go:122] "RemoveContainer" containerID="5926a6fbae2c0783eb6a44082f9d8a6ea6077a61e88281228d98ab99a01c6765"
Oct 30 13:05:13.413152 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:05:13.413127    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5926a6fbae2c0783eb6a44082f9d8a6ea6077a61e88281228d98ab99a01c6765\": container with ID starting with 5926a6fbae2c0783eb6a44082f9d8a6ea6077a61e88281228d98ab99a01c6765 not found: ID does not exist" containerID="5926a6fbae2c0783eb6a44082f9d8a6ea6077a61e88281228d98ab99a01c6765"
Oct 30 13:05:13.413249 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.413166    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5926a6fbae2c0783eb6a44082f9d8a6ea6077a61e88281228d98ab99a01c6765"} err="failed to get container status \"5926a6fbae2c0783eb6a44082f9d8a6ea6077a61e88281228d98ab99a01c6765\": rpc error: code = NotFound desc = could not find container \"5926a6fbae2c0783eb6a44082f9d8a6ea6077a61e88281228d98ab99a01c6765\": container with ID starting with 5926a6fbae2c0783eb6a44082f9d8a6ea6077a61e88281228d98ab99a01c6765 not found: ID does not exist"
Oct 30 13:05:13.413249 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.413185    4945 scope.go:122] "RemoveContainer" containerID="2355652ea89996073cb52d42e22aab8704aa65c36bf021a2061c5c42f882ec73"
Oct 30 13:05:13.422790 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.422775    4945 scope.go:122] "RemoveContainer" containerID="2355652ea89996073cb52d42e22aab8704aa65c36bf021a2061c5c42f882ec73"
Oct 30 13:05:13.423050 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:05:13.423031    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2355652ea89996073cb52d42e22aab8704aa65c36bf021a2061c5c42f882ec73\": container with ID starting with 2355652ea89996073cb52d42e22aab8704aa65c36bf021a2061c5c42f882ec73 not found: ID does not exist" containerID="2355652ea89996073cb52d42e22aab8704aa65c36bf021a2061c5c42f882ec73"
Oct 30 13:05:13.423102 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.423057    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2355652ea89996073cb52d42e22aab8704aa65c36bf021a2061c5c42f882ec73"} err="failed to get container status \"2355652ea89996073cb52d42e22aab8704aa65c36bf021a2061c5c42f882ec73\": rpc error: code = NotFound desc = could not find container \"2355652ea89996073cb52d42e22aab8704aa65c36bf021a2061c5c42f882ec73\": container with ID starting with 2355652ea89996073cb52d42e22aab8704aa65c36bf021a2061c5c42f882ec73 not found: ID does not exist"
Oct 30 13:05:13.452336 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.452311    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"]
Oct 30 13:05:13.463875 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.463854    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-105228/distinct-w2-105228-kserve-5f8d848bdc-bkzmj"]
Oct 30 13:05:13.499169 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.499146    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"]
Oct 30 13:05:13.503118 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.503096    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-105228/distinct-2-w2-105228-kserve-7d7887d596-dbpp4"]
Oct 30 13:05:13.551220 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.551194    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" path="/var/lib/kubelet/pods/699038e0-b6a8-432d-96fb-bea730c1840a/volumes"
Oct 30 13:05:13.551718 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.551700    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" path="/var/lib/kubelet/pods/fbff5ead-c33b-4cce-951d-dc130152f708/volumes"
Oct 30 13:05:13.988104 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.988063    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"]
Oct 30 13:05:13.988317 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:13.988290    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk" podUID="73a5762d-46ba-4f8d-a9c4-89c948315228" containerName="maas-api" containerID="cri-o://f00d4017ee95fd41eafddb89d0574efc7328c53d6d05ae4daa47f3d8401285e0" gracePeriod=30
Oct 30 13:05:14.436122 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:14.435293    4945 generic.go:356] "Generic (PLEG): container finished" podID="73a5762d-46ba-4f8d-a9c4-89c948315228" containerID="f00d4017ee95fd41eafddb89d0574efc7328c53d6d05ae4daa47f3d8401285e0" exitCode=0
Oct 30 13:05:14.436122 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:14.435353    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk" event={"ID":"73a5762d-46ba-4f8d-a9c4-89c948315228","Type":"ContainerDied","Data":"f00d4017ee95fd41eafddb89d0574efc7328c53d6d05ae4daa47f3d8401285e0"}
Oct 30 13:05:14.507819 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:14.507786    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:05:14.680885 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:14.672255    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/73a5762d-46ba-4f8d-a9c4-89c948315228-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73a5762d-46ba-4f8d-a9c4-89c948315228-maas-api-tls\") pod \"73a5762d-46ba-4f8d-a9c4-89c948315228\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") "
Oct 30 13:05:14.680885 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:14.672330    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73a5762d-46ba-4f8d-a9c4-89c948315228-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73a5762d-46ba-4f8d-a9c4-89c948315228-openshift-service-ca\") pod \"73a5762d-46ba-4f8d-a9c4-89c948315228\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") "
Oct 30 13:05:14.680885 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:14.672397    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/73a5762d-46ba-4f8d-a9c4-89c948315228-metrics-certs\" (UniqueName: \"kubernetes.io/secret/73a5762d-46ba-4f8d-a9c4-89c948315228-metrics-certs\") pod \"73a5762d-46ba-4f8d-a9c4-89c948315228\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") "
Oct 30 13:05:14.680885 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:14.672433    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73a5762d-46ba-4f8d-a9c4-89c948315228-kube-api-access-z9cwc\" (UniqueName: \"kubernetes.io/projected/73a5762d-46ba-4f8d-a9c4-89c948315228-kube-api-access-z9cwc\") pod \"73a5762d-46ba-4f8d-a9c4-89c948315228\" (UID: \"73a5762d-46ba-4f8d-a9c4-89c948315228\") "
Oct 30 13:05:14.680885 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:14.674701    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73a5762d-46ba-4f8d-a9c4-89c948315228-openshift-service-ca" pod "73a5762d-46ba-4f8d-a9c4-89c948315228" (UID: "73a5762d-46ba-4f8d-a9c4-89c948315228"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:05:14.688339 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:14.688266    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73a5762d-46ba-4f8d-a9c4-89c948315228-metrics-certs" pod "73a5762d-46ba-4f8d-a9c4-89c948315228" (UID: "73a5762d-46ba-4f8d-a9c4-89c948315228"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:05:14.688889 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:14.688857    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a5762d-46ba-4f8d-a9c4-89c948315228-kube-api-access-z9cwc" pod "73a5762d-46ba-4f8d-a9c4-89c948315228" (UID: "73a5762d-46ba-4f8d-a9c4-89c948315228"). InnerVolumeSpecName "kube-api-access-z9cwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:05:14.689017 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:14.688929    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73a5762d-46ba-4f8d-a9c4-89c948315228-maas-api-tls" pod "73a5762d-46ba-4f8d-a9c4-89c948315228" (UID: "73a5762d-46ba-4f8d-a9c4-89c948315228"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:05:14.773869 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:14.773834    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73a5762d-46ba-4f8d-a9c4-89c948315228-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:14.773869 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:14.773865    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73a5762d-46ba-4f8d-a9c4-89c948315228-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:14.774123 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:14.773878    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73a5762d-46ba-4f8d-a9c4-89c948315228-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:14.774123 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:14.773891    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9cwc\" (UniqueName: \"kubernetes.io/projected/73a5762d-46ba-4f8d-a9c4-89c948315228-kube-api-access-z9cwc\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:15.443872 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:15.443839    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"
Oct 30 13:05:15.444073 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:15.443875    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk" event={"ID":"73a5762d-46ba-4f8d-a9c4-89c948315228","Type":"ContainerDied","Data":"2daeeb125ae9d5b5a7fa00438a20a07f9ba3bf2bffaae5494b2fefed6eedcf42"}
Oct 30 13:05:15.444073 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:15.443949    4945 scope.go:122] "RemoveContainer" containerID="f00d4017ee95fd41eafddb89d0574efc7328c53d6d05ae4daa47f3d8401285e0"
Oct 30 13:05:15.494280 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:15.494243    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"]
Oct 30 13:05:15.506267 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:15.506240    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-ba9a8c-f8d97759c-m88sk"]
Oct 30 13:05:15.551053 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:15.551022    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73a5762d-46ba-4f8d-a9c4-89c948315228" path="/var/lib/kubelet/pods/73a5762d-46ba-4f8d-a9c4-89c948315228/volumes"
Oct 30 13:05:24.264108 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.264071    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"]
Oct 30 13:05:24.264785 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.264763    4945 cpu_manager.go:414] "containerMap: removing container" podUID="368bc543-c9e7-4fd6-a3b3-a147af5b4be3" containerName="maas-api"
Oct 30 13:05:24.264785 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.264782    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="368bc543-c9e7-4fd6-a3b3-a147af5b4be3" containerName="maas-api"
Oct 30 13:05:24.264939 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.264804    4945 cpu_manager.go:414] "containerMap: removing container" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main"
Oct 30 13:05:24.264939 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.264809    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main"
Oct 30 13:05:24.264939 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.264839    4945 cpu_manager.go:414] "containerMap: removing container" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main"
Oct 30 13:05:24.264939 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.264845    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main"
Oct 30 13:05:24.264939 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.264863    4945 cpu_manager.go:414] "containerMap: removing container" podUID="73a5762d-46ba-4f8d-a9c4-89c948315228" containerName="maas-api"
Oct 30 13:05:24.264939 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.264869    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73a5762d-46ba-4f8d-a9c4-89c948315228" containerName="maas-api"
Oct 30 13:05:24.265187 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.264985    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="73a5762d-46ba-4f8d-a9c4-89c948315228" containerName="maas-api"
Oct 30 13:05:24.265187 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.264998    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="368bc543-c9e7-4fd6-a3b3-a147af5b4be3" containerName="maas-api"
Oct 30 13:05:24.265187 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.265005    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbff5ead-c33b-4cce-951d-dc130152f708" containerName="main"
Oct 30 13:05:24.265187 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.265017    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="699038e0-b6a8-432d-96fb-bea730c1840a" containerName="main"
Oct 30 13:05:24.270526 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.270501    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:24.296958 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.296924    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-666577-serving-cert\""
Oct 30 13:05:24.306539 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.306494    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"]
Oct 30 13:05:24.323511 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.323470    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:24.323727 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.323699    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-maas-api-tls\") pod \"maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:24.323879 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.323858    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx6n8\" (UniqueName: \"kubernetes.io/projected/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-kube-api-access-hx6n8\") pod \"maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:24.324005 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.323987    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-metrics-certs\") pod \"maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:24.425944 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.425904    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-maas-api-tls\") pod \"maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:24.426227 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:05:24.426032    4945 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-serving-cert: secret "maas-api-e2e-praxis-b-666577-serving-cert" not found
Oct 30 13:05:24.426352 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.426238    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx6n8\" (UniqueName: \"kubernetes.io/projected/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-kube-api-access-hx6n8\") pod \"maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:24.426352 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:05:24.426288    4945 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-maas-api-tls podName:4ef858d7-259b-4545-ba9f-8e1dd82b8cfd nodeName:}" failed. No retries permitted until 2026-10-30 13:05:24.926264381 +0000 UTC m=+1769.504213451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-maas-api-tls") pod "maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz" (UID: "4ef858d7-259b-4545-ba9f-8e1dd82b8cfd") : secret "maas-api-e2e-praxis-b-666577-serving-cert" not found
Oct 30 13:05:24.426483 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.426385    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-metrics-certs\") pod \"maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:24.426540 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.426512    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:24.427249 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.427226    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:24.428875 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.428856    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-metrics-certs\") pod \"maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:24.446725 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.446704    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx6n8\" (UniqueName: \"kubernetes.io/projected/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-kube-api-access-hx6n8\") pod \"maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:24.934566 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.934504    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-maas-api-tls\") pod \"maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:24.937220 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:24.937195    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-maas-api-tls\") pod \"maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:25.190689 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:25.189900    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:25.588091 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:25.574620    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"]
Oct 30 13:05:26.209172 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:05:26.209128    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:05:26.542821 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:26.542777    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz" event={"ID":"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd","Type":"ContainerStarted","Data":"11557c8dd96fd2858c31744d21a975be78c448ca115381c6db1fa6cce62ada13"}
Oct 30 13:05:26.543025 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:26.542927    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz" event={"ID":"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd","Type":"ContainerStarted","Data":"e7c60f7b715e65275c8d4cbd679b57c25c13cb29900475c84c85cf87e190a586"}
Oct 30 13:05:26.543025 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:26.542980    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:26.587894 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:26.587828    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz" podStartSLOduration=2.171054237 podStartE2EDuration="2.587806839s" totalImagesPullingTime="416.752602ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:05:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:05:26.584516781 +0000 UTC m=+1771.162465871" watchObservedRunningTime="2026-10-30 13:05:26.587806839 +0000 UTC m=+1771.165755928"
Oct 30 13:05:32.555995 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:32.555963    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:05:43.869276 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:43.869240    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"]
Oct 30 13:05:43.869691 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:43.869504    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerName="istio-proxy" containerID="cri-o://807a53e7d415be7f6a80c6cd48990cc8a63a466bafa8b307d6e6a8a0ce1f7b49" gracePeriod=30
Oct 30 13:05:43.881094 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:43.881064    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"]
Oct 30 13:05:43.881316 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:43.881290    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" podUID="a069f35f-76b4-4583-857a-304896b3f33d" containerName="istio-proxy" containerID="cri-o://5dc0da14ca6524d69fb02ff8d95d6e069cf386f9fc824347b453d2fb5bc9d370" gracePeriod=30
Oct 30 13:05:43.897549 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:43.897514    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"]
Oct 30 13:05:43.897795 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:43.897770    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy" containerID="cri-o://b166ec4cf3d0df4978bf46edee0409ea729f486f19d82a86eb5877d51dfa5aca" gracePeriod=30
Oct 30 13:05:46.340098 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.337387    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"]
Oct 30 13:05:46.340098 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.337670    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958" podUID="81e50924-941f-4d0e-a9a3-c7264a448254" containerName="maas-api" containerID="cri-o://2312bf9c056236a1f84c36270900194d7f65fb88b8b34f9402b12387c8aec205" gracePeriod=30
Oct 30 13:05:46.684595 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.684561    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:05:46.748621 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.746917    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81e50924-941f-4d0e-a9a3-c7264a448254-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e50924-941f-4d0e-a9a3-c7264a448254-openshift-service-ca\") pod \"81e50924-941f-4d0e-a9a3-c7264a448254\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") "
Oct 30 13:05:46.748621 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.747097    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81e50924-941f-4d0e-a9a3-c7264a448254-metrics-certs\" (UniqueName: \"kubernetes.io/secret/81e50924-941f-4d0e-a9a3-c7264a448254-metrics-certs\") pod \"81e50924-941f-4d0e-a9a3-c7264a448254\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") "
Oct 30 13:05:46.748621 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.747135    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81e50924-941f-4d0e-a9a3-c7264a448254-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81e50924-941f-4d0e-a9a3-c7264a448254-maas-api-tls\") pod \"81e50924-941f-4d0e-a9a3-c7264a448254\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") "
Oct 30 13:05:46.748621 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.747337    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81e50924-941f-4d0e-a9a3-c7264a448254-kube-api-access-96z6s\" (UniqueName: \"kubernetes.io/projected/81e50924-941f-4d0e-a9a3-c7264a448254-kube-api-access-96z6s\") pod \"81e50924-941f-4d0e-a9a3-c7264a448254\" (UID: \"81e50924-941f-4d0e-a9a3-c7264a448254\") "
Oct 30 13:05:46.756690 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.750983    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81e50924-941f-4d0e-a9a3-c7264a448254-openshift-service-ca" pod "81e50924-941f-4d0e-a9a3-c7264a448254" (UID: "81e50924-941f-4d0e-a9a3-c7264a448254"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:05:46.779104 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.779069    4945 generic.go:356] "Generic (PLEG): container finished" podID="81e50924-941f-4d0e-a9a3-c7264a448254" containerID="2312bf9c056236a1f84c36270900194d7f65fb88b8b34f9402b12387c8aec205" exitCode=0
Oct 30 13:05:46.786637 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.779262    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958" event={"ID":"81e50924-941f-4d0e-a9a3-c7264a448254","Type":"ContainerDied","Data":"2312bf9c056236a1f84c36270900194d7f65fb88b8b34f9402b12387c8aec205"}
Oct 30 13:05:46.786637 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.779326    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958" event={"ID":"81e50924-941f-4d0e-a9a3-c7264a448254","Type":"ContainerDied","Data":"7af5b9fe20849ece7b9838b7651ddbd48812836b66cffd342bb3dba52f72d626"}
Oct 30 13:05:46.786637 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.779356    4945 scope.go:122] "RemoveContainer" containerID="2312bf9c056236a1f84c36270900194d7f65fb88b8b34f9402b12387c8aec205"
Oct 30 13:05:46.786637 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.779530    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"
Oct 30 13:05:46.852620 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.850299    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e50924-941f-4d0e-a9a3-c7264a448254-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:46.887831 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.887790    4945 scope.go:122] "RemoveContainer" containerID="2312bf9c056236a1f84c36270900194d7f65fb88b8b34f9402b12387c8aec205"
Oct 30 13:05:46.891789 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:05:46.891740    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2312bf9c056236a1f84c36270900194d7f65fb88b8b34f9402b12387c8aec205\": container with ID starting with 2312bf9c056236a1f84c36270900194d7f65fb88b8b34f9402b12387c8aec205 not found: ID does not exist" containerID="2312bf9c056236a1f84c36270900194d7f65fb88b8b34f9402b12387c8aec205"
Oct 30 13:05:46.891952 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.891802    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2312bf9c056236a1f84c36270900194d7f65fb88b8b34f9402b12387c8aec205"} err="failed to get container status \"2312bf9c056236a1f84c36270900194d7f65fb88b8b34f9402b12387c8aec205\": rpc error: code = NotFound desc = could not find container \"2312bf9c056236a1f84c36270900194d7f65fb88b8b34f9402b12387c8aec205\": container with ID starting with 2312bf9c056236a1f84c36270900194d7f65fb88b8b34f9402b12387c8aec205 not found: ID does not exist"
Oct 30 13:05:46.903273 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.903172    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81e50924-941f-4d0e-a9a3-c7264a448254-metrics-certs" pod "81e50924-941f-4d0e-a9a3-c7264a448254" (UID: "81e50924-941f-4d0e-a9a3-c7264a448254"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:05:46.903443 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.903417    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81e50924-941f-4d0e-a9a3-c7264a448254-kube-api-access-96z6s" pod "81e50924-941f-4d0e-a9a3-c7264a448254" (UID: "81e50924-941f-4d0e-a9a3-c7264a448254"). InnerVolumeSpecName "kube-api-access-96z6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:05:46.906134 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.906103    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81e50924-941f-4d0e-a9a3-c7264a448254-maas-api-tls" pod "81e50924-941f-4d0e-a9a3-c7264a448254" (UID: "81e50924-941f-4d0e-a9a3-c7264a448254"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:05:46.952019 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.951984    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96z6s\" (UniqueName: \"kubernetes.io/projected/81e50924-941f-4d0e-a9a3-c7264a448254-kube-api-access-96z6s\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:46.952019 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.952016    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81e50924-941f-4d0e-a9a3-c7264a448254-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:46.952225 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:46.952030    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81e50924-941f-4d0e-a9a3-c7264a448254-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:47.149046 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:47.149015    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"]
Oct 30 13:05:47.169504 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:47.169425    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-1f91c1-66b6bb9d85-vf958"]
Oct 30 13:05:47.550403 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:47.550366    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81e50924-941f-4d0e-a9a3-c7264a448254" path="/var/lib/kubelet/pods/81e50924-941f-4d0e-a9a3-c7264a448254/volumes"
Oct 30 13:05:49.826622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:49.815708    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:49.863125 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:49.862286    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 13:05:49.885192 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:49.884972    4945 generic.go:356] "Generic (PLEG): container finished" podID="a069f35f-76b4-4583-857a-304896b3f33d" containerID="5dc0da14ca6524d69fb02ff8d95d6e069cf386f9fc824347b453d2fb5bc9d370" exitCode=0
Oct 30 13:05:49.885192 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:49.885100    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" event={"ID":"a069f35f-76b4-4583-857a-304896b3f33d","Type":"ContainerDied","Data":"5dc0da14ca6524d69fb02ff8d95d6e069cf386f9fc824347b453d2fb5bc9d370"}
Oct 30 13:05:49.891631 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:49.887853    4945 generic.go:356] "Generic (PLEG): container finished" podID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerID="807a53e7d415be7f6a80c6cd48990cc8a63a466bafa8b307d6e6a8a0ce1f7b49" exitCode=0
Oct 30 13:05:49.891631 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:49.887973    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" event={"ID":"81b43fdc-0af8-4443-a1ba-953d6ad604e4","Type":"ContainerDied","Data":"807a53e7d415be7f6a80c6cd48990cc8a63a466bafa8b307d6e6a8a0ce1f7b49"}
Oct 30 13:05:49.905470 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:49.905419    4945 generic.go:356] "Generic (PLEG): container finished" podID="026e8b6e-3faa-4835-9815-489c17d71b82" containerID="b166ec4cf3d0df4978bf46edee0409ea729f486f19d82a86eb5877d51dfa5aca" exitCode=0
Oct 30 13:05:49.905841 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:49.905731    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" event={"ID":"026e8b6e-3faa-4835-9815-489c17d71b82","Type":"ContainerDied","Data":"b166ec4cf3d0df4978bf46edee0409ea729f486f19d82a86eb5877d51dfa5aca"}
Oct 30 13:05:50.140832 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:50.020811    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:50.396859 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:50.396768    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 30 13:05:50.398397 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:50.398361    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 30 13:05:50.407187 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:50.407150    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:50.467047 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:50.467002    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:50.886619 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:50.863486    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 13:05:50.886619 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:50.863877    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 13:05:50.969197 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:50.968449    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 30 13:05:50.969197 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:50.968526    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" podUID="a069f35f-76b4-4583-857a-304896b3f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 30 13:05:51.001568 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:50.984465    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 13:05:51.034633 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.034087    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:51.034633 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.034413    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 13:05:51.296683 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.296199    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" event={"ID":"81b43fdc-0af8-4443-a1ba-953d6ad604e4","Type":"ContainerDied","Data":"a69f97de55635a58542247a81ae842d50d200fd91a9b4daa3be8fca7849e43ac"}
Oct 30 13:05:51.296683 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.296291    4945 scope.go:122] "RemoveContainer" containerID="807a53e7d415be7f6a80c6cd48990cc8a63a466bafa8b307d6e6a8a0ce1f7b49"
Oct 30 13:05:51.317264 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.317056    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"
Oct 30 13:05:51.336827 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.336101    4945 patch_prober.go:29] interesting pod/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 13:05:51.336827 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.336161    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:51.385622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.338616    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-data\") pod \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") "
Oct 30 13:05:51.385622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.338679    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-workload-certs\") pod \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") "
Oct 30 13:05:51.385622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.338768    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-token\" (UniqueName: \"kubernetes.io/projected/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-token\") pod \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") "
Oct 30 13:05:51.385622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.338795    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-workload-socket\") pod \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") "
Oct 30 13:05:51.385622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.338875    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-envoy\") pod \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") "
Oct 30 13:05:51.385622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.338959    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-podinfo\") pod \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") "
Oct 30 13:05:51.385622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.338993    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-credential-socket\") pod \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") "
Oct 30 13:05:51.385622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.339097    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istiod-ca-cert\") pod \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") "
Oct 30 13:05:51.385622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.339148    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81b43fdc-0af8-4443-a1ba-953d6ad604e4-kube-api-access-qd5qq\" (UniqueName: \"kubernetes.io/projected/81b43fdc-0af8-4443-a1ba-953d6ad604e4-kube-api-access-qd5qq\") pod \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\" (UID: \"81b43fdc-0af8-4443-a1ba-953d6ad604e4\") "
Oct 30 13:05:51.385622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.340023    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-workload-socket" pod "81b43fdc-0af8-4443-a1ba-953d6ad604e4" (UID: "81b43fdc-0af8-4443-a1ba-953d6ad604e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:51.385622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.340230    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-data" pod "81b43fdc-0af8-4443-a1ba-953d6ad604e4" (UID: "81b43fdc-0af8-4443-a1ba-953d6ad604e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:51.385622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.340414    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-workload-certs" pod "81b43fdc-0af8-4443-a1ba-953d6ad604e4" (UID: "81b43fdc-0af8-4443-a1ba-953d6ad604e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:51.393623 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.387394    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-credential-socket" pod "81b43fdc-0af8-4443-a1ba-953d6ad604e4" (UID: "81b43fdc-0af8-4443-a1ba-953d6ad604e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:51.423313 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.416689    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:51.423313 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.417182    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istiod-ca-cert" pod "81b43fdc-0af8-4443-a1ba-953d6ad604e4" (UID: "81b43fdc-0af8-4443-a1ba-953d6ad604e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:05:51.456621 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.436723    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:51.457641 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.457589    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 13:05:51.464827 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.464790    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-podinfo" pod "81b43fdc-0af8-4443-a1ba-953d6ad604e4" (UID: "81b43fdc-0af8-4443-a1ba-953d6ad604e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:05:51.492557 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.490710    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:51.492557 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.492002    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 13:05:51.570453 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.521137    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": context deadline exceeded"
Oct 30 13:05:51.664052 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.662402    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-envoy" pod "81b43fdc-0af8-4443-a1ba-953d6ad604e4" (UID: "81b43fdc-0af8-4443-a1ba-953d6ad604e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:51.971794 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.752329    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:51.971794 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.752361    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:51.971794 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.752387    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:51.971794 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:51.752399    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.036787 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.036745    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:52.083266 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.047253    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b43fdc-0af8-4443-a1ba-953d6ad604e4-kube-api-access-qd5qq" pod "81b43fdc-0af8-4443-a1ba-953d6ad604e4" (UID: "81b43fdc-0af8-4443-a1ba-953d6ad604e4"). InnerVolumeSpecName "kube-api-access-qd5qq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:05:52.083266 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.048458    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:52.083266 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.080987    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.083266 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.081016    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.083266 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.081030    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.083266 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.081045    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd5qq\" (UniqueName: \"kubernetes.io/projected/81b43fdc-0af8-4443-a1ba-953d6ad604e4-kube-api-access-qd5qq\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.083266 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.081394    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:52.136876 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.136838    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-token" pod "81b43fdc-0af8-4443-a1ba-953d6ad604e4" (UID: "81b43fdc-0af8-4443-a1ba-953d6ad604e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:05:52.137664 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.137439    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 13:05:52.140758 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.140734    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200039    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a069f35f-76b4-4583-857a-304896b3f33d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a069f35f-76b4-4583-857a-304896b3f33d-istio-podinfo\") pod \"a069f35f-76b4-4583-857a-304896b3f33d\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200124    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/026e8b6e-3faa-4835-9815-489c17d71b82-istio-token\" (UniqueName: \"kubernetes.io/projected/026e8b6e-3faa-4835-9815-489c17d71b82-istio-token\") pod \"026e8b6e-3faa-4835-9815-489c17d71b82\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200169    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-workload-socket\") pod \"a069f35f-76b4-4583-857a-304896b3f33d\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200201    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-workload-socket\") pod \"026e8b6e-3faa-4835-9815-489c17d71b82\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200270    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a069f35f-76b4-4583-857a-304896b3f33d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a069f35f-76b4-4583-857a-304896b3f33d-istiod-ca-cert\") pod \"a069f35f-76b4-4583-857a-304896b3f33d\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200307    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-istio-data\") pod \"a069f35f-76b4-4583-857a-304896b3f33d\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200336    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a069f35f-76b4-4583-857a-304896b3f33d-istio-token\" (UniqueName: \"kubernetes.io/projected/a069f35f-76b4-4583-857a-304896b3f33d-istio-token\") pod \"a069f35f-76b4-4583-857a-304896b3f33d\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200434    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-credential-socket\") pod \"026e8b6e-3faa-4835-9815-489c17d71b82\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200468    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/026e8b6e-3faa-4835-9815-489c17d71b82-kube-api-access-z2x28\" (UniqueName: \"kubernetes.io/projected/026e8b6e-3faa-4835-9815-489c17d71b82-kube-api-access-z2x28\") pod \"026e8b6e-3faa-4835-9815-489c17d71b82\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200534    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/026e8b6e-3faa-4835-9815-489c17d71b82-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/026e8b6e-3faa-4835-9815-489c17d71b82-istiod-ca-cert\") pod \"026e8b6e-3faa-4835-9815-489c17d71b82\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200578    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-workload-certs\") pod \"026e8b6e-3faa-4835-9815-489c17d71b82\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200682    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a069f35f-76b4-4583-857a-304896b3f33d-kube-api-access-rfqmj\" (UniqueName: \"kubernetes.io/projected/a069f35f-76b4-4583-857a-304896b3f33d-kube-api-access-rfqmj\") pod \"a069f35f-76b4-4583-857a-304896b3f33d\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200755    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/026e8b6e-3faa-4835-9815-489c17d71b82-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/026e8b6e-3faa-4835-9815-489c17d71b82-istio-podinfo\") pod \"026e8b6e-3faa-4835-9815-489c17d71b82\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200806    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-istio-envoy\") pod \"026e8b6e-3faa-4835-9815-489c17d71b82\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200833    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-istio-data\") pod \"026e8b6e-3faa-4835-9815-489c17d71b82\" (UID: \"026e8b6e-3faa-4835-9815-489c17d71b82\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200871    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-istio-envoy\") pod \"a069f35f-76b4-4583-857a-304896b3f33d\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200926    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-credential-socket\") pod \"a069f35f-76b4-4583-857a-304896b3f33d\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.200955    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-workload-certs\") pod \"a069f35f-76b4-4583-857a-304896b3f33d\" (UID: \"a069f35f-76b4-4583-857a-304896b3f33d\") "
Oct 30 13:05:52.212625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.201590    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81b43fdc-0af8-4443-a1ba-953d6ad604e4-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.234677 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.227244    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a069f35f-76b4-4583-857a-304896b3f33d-istiod-ca-cert" pod "a069f35f-76b4-4583-857a-304896b3f33d" (UID: "a069f35f-76b4-4583-857a-304896b3f33d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:05:52.235297 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.235117    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-workload-socket" pod "a069f35f-76b4-4583-857a-304896b3f33d" (UID: "a069f35f-76b4-4583-857a-304896b3f33d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:52.235792 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.235494    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-workload-socket" pod "026e8b6e-3faa-4835-9815-489c17d71b82" (UID: "026e8b6e-3faa-4835-9815-489c17d71b82"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:52.236251 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.235957    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-workload-certs" pod "a069f35f-76b4-4583-857a-304896b3f33d" (UID: "a069f35f-76b4-4583-857a-304896b3f33d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:52.263648 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.262245    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-credential-socket" pod "026e8b6e-3faa-4835-9815-489c17d71b82" (UID: "026e8b6e-3faa-4835-9815-489c17d71b82"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:52.264732 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.264473    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/026e8b6e-3faa-4835-9815-489c17d71b82-istiod-ca-cert" pod "026e8b6e-3faa-4835-9815-489c17d71b82" (UID: "026e8b6e-3faa-4835-9815-489c17d71b82"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:05:52.286621 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.265013    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-workload-certs" pod "026e8b6e-3faa-4835-9815-489c17d71b82" (UID: "026e8b6e-3faa-4835-9815-489c17d71b82"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:52.286621 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.284052    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-istio-data" pod "026e8b6e-3faa-4835-9815-489c17d71b82" (UID: "026e8b6e-3faa-4835-9815-489c17d71b82"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:52.286621 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.284423    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a069f35f-76b4-4583-857a-304896b3f33d-kube-api-access-rfqmj" pod "a069f35f-76b4-4583-857a-304896b3f33d" (UID: "a069f35f-76b4-4583-857a-304896b3f33d"). InnerVolumeSpecName "kube-api-access-rfqmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:05:52.293950 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.293716    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-istio-data" pod "a069f35f-76b4-4583-857a-304896b3f33d" (UID: "a069f35f-76b4-4583-857a-304896b3f33d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:52.302821 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.294191    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-credential-socket" pod "a069f35f-76b4-4583-857a-304896b3f33d" (UID: "a069f35f-76b4-4583-857a-304896b3f33d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:52.338624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.308665    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.338624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.308697    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.338624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.308712    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.338624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.308728    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.338624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.308742    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a069f35f-76b4-4583-857a-304896b3f33d-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.338624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.308758    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.338624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.308770    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.338624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.308785    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/026e8b6e-3faa-4835-9815-489c17d71b82-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.338624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.308798    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.338624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.308812    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfqmj\" (UniqueName: \"kubernetes.io/projected/a069f35f-76b4-4583-857a-304896b3f33d-kube-api-access-rfqmj\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.338624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.308826    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.439024 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.437283    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:52.439024 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.437884    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 13:05:52.490029 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.457674    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:52.490029 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.468807    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/026e8b6e-3faa-4835-9815-489c17d71b82-kube-api-access-z2x28" pod "026e8b6e-3faa-4835-9815-489c17d71b82" (UID: "026e8b6e-3faa-4835-9815-489c17d71b82"). InnerVolumeSpecName "kube-api-access-z2x28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:05:52.497648 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.492406    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 13:05:52.500790 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.500730    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-istio-envoy" pod "a069f35f-76b4-4583-857a-304896b3f33d" (UID: "a069f35f-76b4-4583-857a-304896b3f33d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:52.532836 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.521691    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/026e8b6e-3faa-4835-9815-489c17d71b82-istio-podinfo" pod "026e8b6e-3faa-4835-9815-489c17d71b82" (UID: "026e8b6e-3faa-4835-9815-489c17d71b82"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:05:52.532836 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.532786    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a069f35f-76b4-4583-857a-304896b3f33d-istio-token" pod "a069f35f-76b4-4583-857a-304896b3f33d" (UID: "a069f35f-76b4-4583-857a-304896b3f33d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:05:52.665051 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.608727    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-istio-envoy" pod "026e8b6e-3faa-4835-9815-489c17d71b82" (UID: "026e8b6e-3faa-4835-9815-489c17d71b82"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:05:52.665051 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.647379    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a069f35f-76b4-4583-857a-304896b3f33d-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.665051 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.647409    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2x28\" (UniqueName: \"kubernetes.io/projected/026e8b6e-3faa-4835-9815-489c17d71b82-kube-api-access-z2x28\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.665051 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.647424    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/026e8b6e-3faa-4835-9815-489c17d71b82-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.665051 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.647438    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/026e8b6e-3faa-4835-9815-489c17d71b82-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.665051 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.647453    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a069f35f-76b4-4583-857a-304896b3f33d-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:52.929588 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.929543    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a069f35f-76b4-4583-857a-304896b3f33d-istio-podinfo" pod "a069f35f-76b4-4583-857a-304896b3f33d" (UID: "a069f35f-76b4-4583-857a-304896b3f33d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:05:52.939946 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.939898    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w" event={"ID":"026e8b6e-3faa-4835-9815-489c17d71b82","Type":"ContainerDied","Data":"25e116b492d818138e8988b5817c478f27ec2a05bb05cc88f5f3ef0ce27ca710"}
Oct 30 13:05:52.979705 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.943802    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"
Oct 30 13:05:52.991566 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.991533    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"
Oct 30 13:05:52.992911 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.992861    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9" event={"ID":"a069f35f-76b4-4583-857a-304896b3f33d","Type":"ContainerDied","Data":"0a3b4e5b23b49f3bce43d391d6aae7597c98cafa9dad6a31c32a1b7805dd3611"}
Oct 30 13:05:52.993752 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:52.993734    4945 scope.go:122] "RemoveContainer" containerID="b166ec4cf3d0df4978bf46edee0409ea729f486f19d82a86eb5877d51dfa5aca"
Oct 30 13:05:53.007186 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:53.007154    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a069f35f-76b4-4583-857a-304896b3f33d-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:53.040574 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:53.038728    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:53.040574 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:53.039141    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/026e8b6e-3faa-4835-9815-489c17d71b82-istio-token" pod "026e8b6e-3faa-4835-9815-489c17d71b82" (UID: "026e8b6e-3faa-4835-9815-489c17d71b82"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:05:53.135936 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:53.135886    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:53.137693 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:53.137652    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 13:05:53.198625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:53.150204    4945 scope.go:122] "RemoveContainer" containerID="5dc0da14ca6524d69fb02ff8d95d6e069cf386f9fc824347b453d2fb5bc9d370"
Oct 30 13:05:53.289623 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:53.264946    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/026e8b6e-3faa-4835-9815-489c17d71b82-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:05:53.496267 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:53.496217    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:05:53.497342 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:53.497297    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Oct 30 13:05:53.503318 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:53.496484    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 13:05:54.001332 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:53.964546    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:54.044953 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:54.044871    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:54.359766 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:54.356160    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 13:05:54.489650 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:54.485174    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"]
Oct 30 13:05:54.530625 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:05:54.526248    4945 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b43fdc_0af8_4443_a1ba_953d6ad604e4.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda069f35f_76b4_4583_857a_304896b3f33d.slice/crio-0a3b4e5b23b49f3bce43d391d6aae7597c98cafa9dad6a31c32a1b7805dd3611\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b43fdc_0af8_4443_a1ba_953d6ad604e4.slice/crio-a69f97de55635a58542247a81ae842d50d200fd91a9b4daa3be8fca7849e43ac\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026e8b6e_3faa_4835_9815_489c17d71b82.slice/crio-25e116b492d818138e8988b5817c478f27ec2a05bb05cc88f5f3ef0ce27ca710\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda069f35f_76b4_4583_857a_304896b3f33d.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026e8b6e_3faa_4835_9815_489c17d71b82.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 13:05:54.530625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:54.529674    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:05:54.551422 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:05:54.551365    4945 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda069f35f_76b4_4583_857a_304896b3f33d.slice/crio-0a3b4e5b23b49f3bce43d391d6aae7597c98cafa9dad6a31c32a1b7805dd3611\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026e8b6e_3faa_4835_9815_489c17d71b82.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda069f35f_76b4_4583_857a_304896b3f33d.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod026e8b6e_3faa_4835_9815_489c17d71b82.slice/crio-25e116b492d818138e8988b5817c478f27ec2a05bb05cc88f5f3ef0ce27ca710\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b43fdc_0af8_4443_a1ba_953d6ad604e4.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b43fdc_0af8_4443_a1ba_953d6ad604e4.slice/crio-a69f97de55635a58542247a81ae842d50d200fd91a9b4daa3be8fca7849e43ac\": RecentStats: unable to find data in memory cache]"
Oct 30 13:05:54.610841 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:54.610744    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-t6qs9"]
Oct 30 13:05:54.655630 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:05:54.641362    4945 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b43fdc_0af8_4443_a1ba_953d6ad604e4.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda069f35f_76b4_4583_857a_304896b3f33d.slice/crio-0a3b4e5b23b49f3bce43d391d6aae7597c98cafa9dad6a31c32a1b7805dd3611\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b43fdc_0af8_4443_a1ba_953d6ad604e4.slice/crio-a69f97de55635a58542247a81ae842d50d200fd91a9b4daa3be8fca7849e43ac\": RecentStats: unable to find data in memory cache]"
Oct 30 13:05:54.666794 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:54.657168    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 13:05:54.667066 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:54.498116    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:54.722937 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:54.722893    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"]
Oct 30 13:05:54.843621 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:54.839656    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-z9qv9"]
Oct 30 13:05:55.162625 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:55.161494    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:55.256711 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:55.256671    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"]
Oct 30 13:05:55.375550 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:55.375498    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 13:05:55.456679 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:55.413456    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-105228-openshift-default-7cfc8cdfb5-kcd5w"]
Oct 30 13:05:55.479624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:55.460941    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 13:05:55.595637 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:55.572014    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded"
Oct 30 13:05:55.698739 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:55.698690    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:05:55.699367 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:55.699332    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Oct 30 13:05:56.043623 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:56.037835    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" path="/var/lib/kubelet/pods/026e8b6e-3faa-4835-9815-489c17d71b82/volumes"
Oct 30 13:05:56.211136 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:56.211099    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" path="/var/lib/kubelet/pods/81b43fdc-0af8-4443-a1ba-953d6ad604e4/volumes"
Oct 30 13:05:56.221436 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:56.221392    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a069f35f-76b4-4583-857a-304896b3f33d" path="/var/lib/kubelet/pods/a069f35f-76b4-4583-857a-304896b3f33d/volumes"
Oct 30 13:05:56.229880 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:56.229853    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 13:05:56.240738 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:56.239050    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 13:05:56.242942 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:56.242924    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 13:05:56.250648 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:56.250162    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 13:05:56.264082 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:05:56.264051    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:05:56.350542 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:56.350464    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 13:05:56.351075 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:05:56.351051    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 13:06:06.725758 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:06.725719    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"]
Oct 30 13:06:06.726216 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:06.725982    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td" podUID="55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e" containerName="maas-api" containerID="cri-o://591e2b77d5762913353bf21fb6e26e5a7b1026d189fc7f678b62aabe1f1a4099" gracePeriod=30
Oct 30 13:06:06.866291 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:06.866015    4945 generic.go:356] "Generic (PLEG): container finished" podID="55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e" containerID="591e2b77d5762913353bf21fb6e26e5a7b1026d189fc7f678b62aabe1f1a4099" exitCode=0
Oct 30 13:06:06.866291 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:06.866172    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td" event={"ID":"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e","Type":"ContainerDied","Data":"591e2b77d5762913353bf21fb6e26e5a7b1026d189fc7f678b62aabe1f1a4099"}
Oct 30 13:06:07.255404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.255081    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:06:07.353463 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.353052    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-metrics-certs\") pod \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") "
Oct 30 13:06:07.353792 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.353768    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-maas-api-tls\") pod \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") "
Oct 30 13:06:07.354079 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.354061    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-openshift-service-ca\") pod \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") "
Oct 30 13:06:07.354397 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.354375    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-kube-api-access-s5fhz\" (UniqueName: \"kubernetes.io/projected/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-kube-api-access-s5fhz\") pod \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\" (UID: \"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e\") "
Oct 30 13:06:07.354572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.354512    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-openshift-service-ca" pod "55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e" (UID: "55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:06:07.355215 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.355191    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:06:07.357306 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.357263    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-kube-api-access-s5fhz" pod "55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e" (UID: "55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e"). InnerVolumeSpecName "kube-api-access-s5fhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:06:07.357516 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.357496    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-metrics-certs" pod "55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e" (UID: "55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:06:07.357690 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.357660    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-maas-api-tls" pod "55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e" (UID: "55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:06:07.456218 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.456186    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:06:07.456218 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.456213    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:06:07.456218 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.456224    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5fhz\" (UniqueName: \"kubernetes.io/projected/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e-kube-api-access-s5fhz\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:06:07.877714 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.877626    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td" event={"ID":"55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e","Type":"ContainerDied","Data":"65ea8a6783ed5387a5e47cc1b0982cf3fa3e64b6834c6805734c6bd1bea62996"}
Oct 30 13:06:07.877714 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.877709    4945 scope.go:122] "RemoveContainer" containerID="591e2b77d5762913353bf21fb6e26e5a7b1026d189fc7f678b62aabe1f1a4099"
Oct 30 13:06:07.878237 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.877902    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"
Oct 30 13:06:07.930729 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.930696    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"]
Oct 30 13:06:07.941225 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:07.941200    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-f73c9f-588b76499c-cq9td"]
Oct 30 13:06:09.550662 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:09.550623    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e" path="/var/lib/kubelet/pods/55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e/volumes"
Oct 30 13:06:26.283624 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:06:26.282817    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:06:33.012221 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:33.010268    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"]
Oct 30 13:06:33.012221 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:33.010545    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz" podUID="4ef858d7-259b-4545-ba9f-8e1dd82b8cfd" containerName="maas-api" containerID="cri-o://11557c8dd96fd2858c31744d21a975be78c448ca115381c6db1fa6cce62ada13" gracePeriod=30
Oct 30 13:06:33.491831 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:33.488255    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:06:33.606118 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:33.606067    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-kube-api-access-hx6n8\" (UniqueName: \"kubernetes.io/projected/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-kube-api-access-hx6n8\") pod \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") "
Oct 30 13:06:33.606506 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:33.606480    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-metrics-certs\") pod \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") "
Oct 30 13:06:33.606652 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:33.606635    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-maas-api-tls\") pod \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") "
Oct 30 13:06:33.606776 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:33.606706    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-openshift-service-ca\") pod \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\" (UID: \"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd\") "
Oct 30 13:06:33.611852 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:33.611799    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-openshift-service-ca" pod "4ef858d7-259b-4545-ba9f-8e1dd82b8cfd" (UID: "4ef858d7-259b-4545-ba9f-8e1dd82b8cfd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:06:33.615804 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:33.615770    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-metrics-certs" pod "4ef858d7-259b-4545-ba9f-8e1dd82b8cfd" (UID: "4ef858d7-259b-4545-ba9f-8e1dd82b8cfd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:06:33.616760 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:33.616732    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-kube-api-access-hx6n8" pod "4ef858d7-259b-4545-ba9f-8e1dd82b8cfd" (UID: "4ef858d7-259b-4545-ba9f-8e1dd82b8cfd"). InnerVolumeSpecName "kube-api-access-hx6n8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:06:33.617528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:33.617493    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-maas-api-tls" pod "4ef858d7-259b-4545-ba9f-8e1dd82b8cfd" (UID: "4ef858d7-259b-4545-ba9f-8e1dd82b8cfd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:06:33.711735 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:33.711699    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:06:33.711735 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:33.711729    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:06:33.711735 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:33.711739    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:06:33.712039 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:33.711750    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx6n8\" (UniqueName: \"kubernetes.io/projected/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd-kube-api-access-hx6n8\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:06:34.089406 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:34.089311    4945 generic.go:356] "Generic (PLEG): container finished" podID="4ef858d7-259b-4545-ba9f-8e1dd82b8cfd" containerID="11557c8dd96fd2858c31744d21a975be78c448ca115381c6db1fa6cce62ada13" exitCode=0
Oct 30 13:06:34.089406 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:34.089368    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"
Oct 30 13:06:34.089935 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:34.089401    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz" event={"ID":"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd","Type":"ContainerDied","Data":"11557c8dd96fd2858c31744d21a975be78c448ca115381c6db1fa6cce62ada13"}
Oct 30 13:06:34.089935 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:34.089475    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz" event={"ID":"4ef858d7-259b-4545-ba9f-8e1dd82b8cfd","Type":"ContainerDied","Data":"e7c60f7b715e65275c8d4cbd679b57c25c13cb29900475c84c85cf87e190a586"}
Oct 30 13:06:34.089935 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:34.089510    4945 scope.go:122] "RemoveContainer" containerID="11557c8dd96fd2858c31744d21a975be78c448ca115381c6db1fa6cce62ada13"
Oct 30 13:06:34.107301 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:34.107270    4945 scope.go:122] "RemoveContainer" containerID="11557c8dd96fd2858c31744d21a975be78c448ca115381c6db1fa6cce62ada13"
Oct 30 13:06:34.107701 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:06:34.107671    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11557c8dd96fd2858c31744d21a975be78c448ca115381c6db1fa6cce62ada13\": container with ID starting with 11557c8dd96fd2858c31744d21a975be78c448ca115381c6db1fa6cce62ada13 not found: ID does not exist" containerID="11557c8dd96fd2858c31744d21a975be78c448ca115381c6db1fa6cce62ada13"
Oct 30 13:06:34.107783 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:34.107714    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11557c8dd96fd2858c31744d21a975be78c448ca115381c6db1fa6cce62ada13"} err="failed to get container status \"11557c8dd96fd2858c31744d21a975be78c448ca115381c6db1fa6cce62ada13\": rpc error: code = NotFound desc = could not find container \"11557c8dd96fd2858c31744d21a975be78c448ca115381c6db1fa6cce62ada13\": container with ID starting with 11557c8dd96fd2858c31744d21a975be78c448ca115381c6db1fa6cce62ada13 not found: ID does not exist"
Oct 30 13:06:34.143267 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:34.143236    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"]
Oct 30 13:06:34.154174 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:34.154148    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-666577-64dbb787ff-9n7dz"]
Oct 30 13:06:35.550764 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:35.550730    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4ef858d7-259b-4545-ba9f-8e1dd82b8cfd" path="/var/lib/kubelet/pods/4ef858d7-259b-4545-ba9f-8e1dd82b8cfd/volumes"
Oct 30 13:06:46.464709 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:46.464662    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"]
Oct 30 13:06:46.478065 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:06:46.478009    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"]
Oct 30 13:06:56.405773 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:06:56.405721    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:07:01.491061 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.491016    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main" containerID="cri-o://c87ca46100e3384d9043ea7b67da5057f9aa2a8655556b09794129ac8a65e31b" gracePeriod=45
Oct 30 13:07:01.500644 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.500566    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" containerID="cri-o://b04432ed907e257cc1efd069344226762166c83900ad06797c1be3eb7f015d9b" gracePeriod=45
Oct 30 13:07:01.504813 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.504785    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": dial tcp 10.132.0.60:8000: connect: connection refused"
Oct 30 13:07:01.823342 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.823306    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 13:07:01.871337 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.871306    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 13:07:01.972847 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.970012    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-model-cache\") pod \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") "
Oct 30 13:07:01.972847 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.970101    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-tmp-dir\") pod \"b41d6c6e-7427-499e-b4b5-b69590e5a458\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") "
Oct 30 13:07:01.972847 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.970162    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-tls-certs\" (UniqueName: \"kubernetes.io/secret/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-tls-certs\") pod \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") "
Oct 30 13:07:01.972847 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.970240    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-home\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-home\") pod \"b41d6c6e-7427-499e-b4b5-b69590e5a458\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") "
Oct 30 13:07:01.972847 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.970271    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b41d6c6e-7427-499e-b4b5-b69590e5a458-kube-api-access-ftlwg\" (UniqueName: \"kubernetes.io/projected/b41d6c6e-7427-499e-b4b5-b69590e5a458-kube-api-access-ftlwg\") pod \"b41d6c6e-7427-499e-b4b5-b69590e5a458\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") "
Oct 30 13:07:01.972847 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.970307    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-model-cache\") pod \"b41d6c6e-7427-499e-b4b5-b69590e5a458\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") "
Oct 30 13:07:01.972847 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.970392    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-tmp-dir\") pod \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") "
Oct 30 13:07:01.972847 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.970398    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-tmp-dir" pod "b41d6c6e-7427-499e-b4b5-b69590e5a458" (UID: "b41d6c6e-7427-499e-b4b5-b69590e5a458"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:01.972847 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.970441    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-dshm\") pod \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") "
Oct 30 13:07:01.972847 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.970469    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-dshm\") pod \"b41d6c6e-7427-499e-b4b5-b69590e5a458\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") "
Oct 30 13:07:01.972847 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.970496    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-home\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-home\") pod \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") "
Oct 30 13:07:01.972847 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.970620    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-home" pod "b41d6c6e-7427-499e-b4b5-b69590e5a458" (UID: "b41d6c6e-7427-499e-b4b5-b69590e5a458"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:01.972847 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.970626    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b41d6c6e-7427-499e-b4b5-b69590e5a458-tls-certs\" (UniqueName: \"kubernetes.io/secret/b41d6c6e-7427-499e-b4b5-b69590e5a458-tls-certs\") pod \"b41d6c6e-7427-499e-b4b5-b69590e5a458\" (UID: \"b41d6c6e-7427-499e-b4b5-b69590e5a458\") "
Oct 30 13:07:01.972847 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.970722    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-kube-api-access-xbbq7\" (UniqueName: \"kubernetes.io/projected/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-kube-api-access-xbbq7\") pod \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\" (UID: \"819b89b0-600a-4bb8-8a5c-adaa0d941c7e\") "
Oct 30 13:07:01.977922 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.977888    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-model-cache" pod "819b89b0-600a-4bb8-8a5c-adaa0d941c7e" (UID: "819b89b0-600a-4bb8-8a5c-adaa0d941c7e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:01.978101 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.978082    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-tmp-dir" pod "819b89b0-600a-4bb8-8a5c-adaa0d941c7e" (UID: "819b89b0-600a-4bb8-8a5c-adaa0d941c7e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:01.978724 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.978666    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-model-cache" pod "b41d6c6e-7427-499e-b4b5-b69590e5a458" (UID: "b41d6c6e-7427-499e-b4b5-b69590e5a458"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:01.978898 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.978878    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-home" pod "819b89b0-600a-4bb8-8a5c-adaa0d941c7e" (UID: "819b89b0-600a-4bb8-8a5c-adaa0d941c7e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:01.982258 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.982236    4945 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-tmp-dir\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:01.982258 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.982259    4945 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-home\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:01.982499 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.982272    4945 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-model-cache\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:01.982499 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.982284    4945 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-tmp-dir\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:01.982499 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.982295    4945 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-home\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:01.982499 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.982307    4945 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-model-cache\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:01.983165 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.983111    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-dshm" pod "b41d6c6e-7427-499e-b4b5-b69590e5a458" (UID: "b41d6c6e-7427-499e-b4b5-b69590e5a458"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:01.983234 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.983175    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-dshm" pod "819b89b0-600a-4bb8-8a5c-adaa0d941c7e" (UID: "819b89b0-600a-4bb8-8a5c-adaa0d941c7e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:01.983328 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.983258    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b41d6c6e-7427-499e-b4b5-b69590e5a458-kube-api-access-ftlwg" pod "b41d6c6e-7427-499e-b4b5-b69590e5a458" (UID: "b41d6c6e-7427-499e-b4b5-b69590e5a458"). InnerVolumeSpecName "kube-api-access-ftlwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:01.984609 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.984559    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-kube-api-access-xbbq7" pod "819b89b0-600a-4bb8-8a5c-adaa0d941c7e" (UID: "819b89b0-600a-4bb8-8a5c-adaa0d941c7e"). InnerVolumeSpecName "kube-api-access-xbbq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:01.984772 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.984749    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-tls-certs" pod "819b89b0-600a-4bb8-8a5c-adaa0d941c7e" (UID: "819b89b0-600a-4bb8-8a5c-adaa0d941c7e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:07:01.984985 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:01.984962    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b41d6c6e-7427-499e-b4b5-b69590e5a458-tls-certs" pod "b41d6c6e-7427-499e-b4b5-b69590e5a458" (UID: "b41d6c6e-7427-499e-b4b5-b69590e5a458"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:07:02.092675 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.088699    4945 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b41d6c6e-7427-499e-b4b5-b69590e5a458-tls-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:02.092675 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.088736    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbbq7\" (UniqueName: \"kubernetes.io/projected/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-kube-api-access-xbbq7\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:02.092675 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.088752    4945 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-tls-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:02.092675 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.088766    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftlwg\" (UniqueName: \"kubernetes.io/projected/b41d6c6e-7427-499e-b4b5-b69590e5a458-kube-api-access-ftlwg\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:02.092675 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.088778    4945 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/819b89b0-600a-4bb8-8a5c-adaa0d941c7e-dshm\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:02.092675 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.088791    4945 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b41d6c6e-7427-499e-b4b5-b69590e5a458-dshm\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:02.447063 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.447031    4945 generic.go:356] "Generic (PLEG): container finished" podID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerID="c87ca46100e3384d9043ea7b67da5057f9aa2a8655556b09794129ac8a65e31b" exitCode=0
Oct 30 13:07:02.447269 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.447075    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"
Oct 30 13:07:02.447269 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.447121    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" event={"ID":"819b89b0-600a-4bb8-8a5c-adaa0d941c7e","Type":"ContainerDied","Data":"c87ca46100e3384d9043ea7b67da5057f9aa2a8655556b09794129ac8a65e31b"}
Oct 30 13:07:02.447269 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.447164    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6" event={"ID":"819b89b0-600a-4bb8-8a5c-adaa0d941c7e","Type":"ContainerDied","Data":"478639405ba5b2ad38aede293777d1ab297a00ab5ec26e7bc9569ed661c1fe26"}
Oct 30 13:07:02.447269 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.447187    4945 scope.go:122] "RemoveContainer" containerID="c87ca46100e3384d9043ea7b67da5057f9aa2a8655556b09794129ac8a65e31b"
Oct 30 13:07:02.448681 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.448664    4945 generic.go:356] "Generic (PLEG): container finished" podID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerID="b04432ed907e257cc1efd069344226762166c83900ad06797c1be3eb7f015d9b" exitCode=0
Oct 30 13:07:02.448760 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.448745    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"
Oct 30 13:07:02.448818 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.448769    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" event={"ID":"b41d6c6e-7427-499e-b4b5-b69590e5a458","Type":"ContainerDied","Data":"b04432ed907e257cc1efd069344226762166c83900ad06797c1be3eb7f015d9b"}
Oct 30 13:07:02.448818 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.448800    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2" event={"ID":"b41d6c6e-7427-499e-b4b5-b69590e5a458","Type":"ContainerDied","Data":"257f5ebe2e9e9c4778f2e02c0ba881902b12c0c2031a318207f41bd3ccc15e8a"}
Oct 30 13:07:02.461268 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.461246    4945 scope.go:122] "RemoveContainer" containerID="c87ca46100e3384d9043ea7b67da5057f9aa2a8655556b09794129ac8a65e31b"
Oct 30 13:07:02.461590 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:07:02.461569    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c87ca46100e3384d9043ea7b67da5057f9aa2a8655556b09794129ac8a65e31b\": container with ID starting with c87ca46100e3384d9043ea7b67da5057f9aa2a8655556b09794129ac8a65e31b not found: ID does not exist" containerID="c87ca46100e3384d9043ea7b67da5057f9aa2a8655556b09794129ac8a65e31b"
Oct 30 13:07:02.461729 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.461614    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c87ca46100e3384d9043ea7b67da5057f9aa2a8655556b09794129ac8a65e31b"} err="failed to get container status \"c87ca46100e3384d9043ea7b67da5057f9aa2a8655556b09794129ac8a65e31b\": rpc error: code = NotFound desc = could not find container \"c87ca46100e3384d9043ea7b67da5057f9aa2a8655556b09794129ac8a65e31b\": container with ID starting with c87ca46100e3384d9043ea7b67da5057f9aa2a8655556b09794129ac8a65e31b not found: ID does not exist"
Oct 30 13:07:02.461729 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.461640    4945 scope.go:122] "RemoveContainer" containerID="b04432ed907e257cc1efd069344226762166c83900ad06797c1be3eb7f015d9b"
Oct 30 13:07:02.490613 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.480759    4945 scope.go:122] "RemoveContainer" containerID="b04432ed907e257cc1efd069344226762166c83900ad06797c1be3eb7f015d9b"
Oct 30 13:07:02.490613 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:07:02.484353    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b04432ed907e257cc1efd069344226762166c83900ad06797c1be3eb7f015d9b\": container with ID starting with b04432ed907e257cc1efd069344226762166c83900ad06797c1be3eb7f015d9b not found: ID does not exist" containerID="b04432ed907e257cc1efd069344226762166c83900ad06797c1be3eb7f015d9b"
Oct 30 13:07:02.490613 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.484390    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b04432ed907e257cc1efd069344226762166c83900ad06797c1be3eb7f015d9b"} err="failed to get container status \"b04432ed907e257cc1efd069344226762166c83900ad06797c1be3eb7f015d9b\": rpc error: code = NotFound desc = could not find container \"b04432ed907e257cc1efd069344226762166c83900ad06797c1be3eb7f015d9b\": container with ID starting with b04432ed907e257cc1efd069344226762166c83900ad06797c1be3eb7f015d9b not found: ID does not exist"
Oct 30 13:07:02.518907 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.517429    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"]
Oct 30 13:07:02.526264 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.526238    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-4cd3fb/premium-w0-4cd3fb-kserve-6696787bf8-dxpl6"]
Oct 30 13:07:02.564134 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.564099    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"]
Oct 30 13:07:02.567508 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:02.567484    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-4cd3fb/simulator-w0-4cd3fb-kserve-6ffbdc64d7-zpzt2"]
Oct 30 13:07:03.554903 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:03.554861    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" path="/var/lib/kubelet/pods/819b89b0-600a-4bb8-8a5c-adaa0d941c7e/volumes"
Oct 30 13:07:03.555544 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:03.555521    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" path="/var/lib/kubelet/pods/b41d6c6e-7427-499e-b4b5-b69590e5a458/volumes"
Oct 30 13:07:05.781624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:05.775650    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"]
Oct 30 13:07:05.781624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:05.775930    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt" podUID="579ade75-eb45-4c2b-add1-d199586065c0" containerName="istio-proxy" containerID="cri-o://595cf7c70eabfad4fbf8790bb1bf32c8f4c9008fb8c061dcfd5aa58281924055" gracePeriod=30
Oct 30 13:07:05.786765 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:05.786736    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"]
Oct 30 13:07:05.787233 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:05.787179    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" podUID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" containerName="istio-proxy" containerID="cri-o://0149777bc760a9584c3f6ec950064b7e65f3cf1829d6af7b991f5f5d2b2c780f" gracePeriod=30
Oct 30 13:07:05.793296 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:05.793274    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"]
Oct 30 13:07:05.793800 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:05.793768    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr" podUID="facb034f-f30c-4290-8ad3-07b58b7163e0" containerName="istio-proxy" containerID="cri-o://ce68dc9dab1c95a224b9fb561d51f0aa8dbe3f3ea2f94af8b8e1c1612a09874e" gracePeriod=30
Oct 30 13:07:07.090622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:07.089687    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 30 13:07:07.288623 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:07.270410    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:07:08.106624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:08.091827    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:07:08.106624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:08.095031    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 13:07:08.300338 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:08.300274    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:07:08.525989 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:08.525664    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:07:08.541353 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:08.541320    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 13:07:08.552647 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:08.542936    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 13:07:08.584609 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:08.566343    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 13:07:10.292628 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.292577    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"]
Oct 30 13:07:10.293154 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.292834    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx" podUID="120ca3bc-b398-43fa-b2d1-1dfa81002f9c" containerName="maas-api" containerID="cri-o://f6007c3b8cd5cc4d0aaa187a3c62c62ece682dd02f99ff63d3094361775f248c" gracePeriod=30
Oct 30 13:07:10.347906 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.347856    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx" podUID="120ca3bc-b398-43fa-b2d1-1dfa81002f9c" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.59:8443/health\": dial tcp 10.132.0.59:8443: connect: connection refused"
Oct 30 13:07:10.750029 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.749989    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 13:07:10.835124 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.835056    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-openshift-service-ca\") pod \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") "
Oct 30 13:07:10.835318 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.835201    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-kube-api-access-s8qwx\" (UniqueName: \"kubernetes.io/projected/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-kube-api-access-s8qwx\") pod \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") "
Oct 30 13:07:10.835392 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.835344    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-metrics-certs\") pod \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") "
Oct 30 13:07:10.838619 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.835559    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-maas-api-tls\") pod \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\" (UID: \"120ca3bc-b398-43fa-b2d1-1dfa81002f9c\") "
Oct 30 13:07:10.838752 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.838697    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-openshift-service-ca" pod "120ca3bc-b398-43fa-b2d1-1dfa81002f9c" (UID: "120ca3bc-b398-43fa-b2d1-1dfa81002f9c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:07:10.848488 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.848456    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-metrics-certs" pod "120ca3bc-b398-43fa-b2d1-1dfa81002f9c" (UID: "120ca3bc-b398-43fa-b2d1-1dfa81002f9c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:07:10.954966 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.946252    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:10.954966 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.946287    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:10.954966 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.952890    4945 generic.go:356] "Generic (PLEG): container finished" podID="120ca3bc-b398-43fa-b2d1-1dfa81002f9c" containerID="f6007c3b8cd5cc4d0aaa187a3c62c62ece682dd02f99ff63d3094361775f248c" exitCode=0
Oct 30 13:07:10.954966 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.953158    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"
Oct 30 13:07:10.954966 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.953632    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx" event={"ID":"120ca3bc-b398-43fa-b2d1-1dfa81002f9c","Type":"ContainerDied","Data":"f6007c3b8cd5cc4d0aaa187a3c62c62ece682dd02f99ff63d3094361775f248c"}
Oct 30 13:07:10.954966 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.953698    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx" event={"ID":"120ca3bc-b398-43fa-b2d1-1dfa81002f9c","Type":"ContainerDied","Data":"5240e230a88baecebce2552a83a81049735c291627d167c8c8a257b4ef097a57"}
Oct 30 13:07:10.954966 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.953729    4945 scope.go:122] "RemoveContainer" containerID="f6007c3b8cd5cc4d0aaa187a3c62c62ece682dd02f99ff63d3094361775f248c"
Oct 30 13:07:10.961797 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.961572    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-maas-api-tls" pod "120ca3bc-b398-43fa-b2d1-1dfa81002f9c" (UID: "120ca3bc-b398-43fa-b2d1-1dfa81002f9c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:07:10.964080 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:10.964054    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-kube-api-access-s8qwx" pod "120ca3bc-b398-43fa-b2d1-1dfa81002f9c" (UID: "120ca3bc-b398-43fa-b2d1-1dfa81002f9c"). InnerVolumeSpecName "kube-api-access-s8qwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:11.026289 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.025745    4945 scope.go:122] "RemoveContainer" containerID="f6007c3b8cd5cc4d0aaa187a3c62c62ece682dd02f99ff63d3094361775f248c"
Oct 30 13:07:11.026996 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:07:11.026541    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6007c3b8cd5cc4d0aaa187a3c62c62ece682dd02f99ff63d3094361775f248c\": container with ID starting with f6007c3b8cd5cc4d0aaa187a3c62c62ece682dd02f99ff63d3094361775f248c not found: ID does not exist" containerID="f6007c3b8cd5cc4d0aaa187a3c62c62ece682dd02f99ff63d3094361775f248c"
Oct 30 13:07:11.027113 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.026580    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6007c3b8cd5cc4d0aaa187a3c62c62ece682dd02f99ff63d3094361775f248c"} err="failed to get container status \"f6007c3b8cd5cc4d0aaa187a3c62c62ece682dd02f99ff63d3094361775f248c\": rpc error: code = NotFound desc = could not find container \"f6007c3b8cd5cc4d0aaa187a3c62c62ece682dd02f99ff63d3094361775f248c\": container with ID starting with f6007c3b8cd5cc4d0aaa187a3c62c62ece682dd02f99ff63d3094361775f248c not found: ID does not exist"
Oct 30 13:07:11.048678 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.048254    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.048678 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.048292    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8qwx\" (UniqueName: \"kubernetes.io/projected/120ca3bc-b398-43fa-b2d1-1dfa81002f9c-kube-api-access-s8qwx\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.222644 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.222340    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 13:07:11.263566 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.263529    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 13:07:11.342016 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.335506    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"]
Oct 30 13:07:11.342016 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.339444    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-4cd3fb-8679f48966-xjnhx"]
Oct 30 13:07:11.353486 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.353454    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/579ade75-eb45-4c2b-add1-d199586065c0-istio-token\" (UniqueName: \"kubernetes.io/projected/579ade75-eb45-4c2b-add1-d199586065c0-istio-token\") pod \"579ade75-eb45-4c2b-add1-d199586065c0\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") "
Oct 30 13:07:11.353768 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.353744    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-podinfo\") pod \"facb034f-f30c-4290-8ad3-07b58b7163e0\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") "
Oct 30 13:07:11.354412 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.354391    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-data\") pod \"facb034f-f30c-4290-8ad3-07b58b7163e0\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") "
Oct 30 13:07:11.356541 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.356485    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-credential-socket\") pod \"facb034f-f30c-4290-8ad3-07b58b7163e0\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") "
Oct 30 13:07:11.356759 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.356741    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-workload-socket\") pod \"579ade75-eb45-4c2b-add1-d199586065c0\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") "
Oct 30 13:07:11.356869 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.356839    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-credential-socket" pod "facb034f-f30c-4290-8ad3-07b58b7163e0" (UID: "facb034f-f30c-4290-8ad3-07b58b7163e0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:11.357000 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.356981    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-workload-socket\") pod \"facb034f-f30c-4290-8ad3-07b58b7163e0\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") "
Oct 30 13:07:11.357120 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.357105    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-istio-envoy\") pod \"579ade75-eb45-4c2b-add1-d199586065c0\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") "
Oct 30 13:07:11.357242 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.357226    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-istio-data\") pod \"579ade75-eb45-4c2b-add1-d199586065c0\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") "
Oct 30 13:07:11.357366 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.357350    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-token\" (UniqueName: \"kubernetes.io/projected/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-token\") pod \"facb034f-f30c-4290-8ad3-07b58b7163e0\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") "
Oct 30 13:07:11.357494 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.357478    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/579ade75-eb45-4c2b-add1-d199586065c0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/579ade75-eb45-4c2b-add1-d199586065c0-istio-podinfo\") pod \"579ade75-eb45-4c2b-add1-d199586065c0\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") "
Oct 30 13:07:11.357665 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.357634    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/579ade75-eb45-4c2b-add1-d199586065c0-kube-api-access-j45kf\" (UniqueName: \"kubernetes.io/projected/579ade75-eb45-4c2b-add1-d199586065c0-kube-api-access-j45kf\") pod \"579ade75-eb45-4c2b-add1-d199586065c0\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") "
Oct 30 13:07:11.357838 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.357820    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-workload-certs\") pod \"579ade75-eb45-4c2b-add1-d199586065c0\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") "
Oct 30 13:07:11.357998 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.357982    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/facb034f-f30c-4290-8ad3-07b58b7163e0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/facb034f-f30c-4290-8ad3-07b58b7163e0-istiod-ca-cert\") pod \"facb034f-f30c-4290-8ad3-07b58b7163e0\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") "
Oct 30 13:07:11.358156 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.358140    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-envoy\") pod \"facb034f-f30c-4290-8ad3-07b58b7163e0\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") "
Oct 30 13:07:11.358292 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.358277    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/facb034f-f30c-4290-8ad3-07b58b7163e0-kube-api-access-f9r8z\" (UniqueName: \"kubernetes.io/projected/facb034f-f30c-4290-8ad3-07b58b7163e0-kube-api-access-f9r8z\") pod \"facb034f-f30c-4290-8ad3-07b58b7163e0\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") "
Oct 30 13:07:11.358394 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.358374    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-data" pod "facb034f-f30c-4290-8ad3-07b58b7163e0" (UID: "facb034f-f30c-4290-8ad3-07b58b7163e0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:11.358526 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.358509    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-credential-socket\") pod \"579ade75-eb45-4c2b-add1-d199586065c0\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") "
Oct 30 13:07:11.359480 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.359401    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-workload-certs\") pod \"facb034f-f30c-4290-8ad3-07b58b7163e0\" (UID: \"facb034f-f30c-4290-8ad3-07b58b7163e0\") "
Oct 30 13:07:11.359480 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.359476    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/579ade75-eb45-4c2b-add1-d199586065c0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/579ade75-eb45-4c2b-add1-d199586065c0-istiod-ca-cert\") pod \"579ade75-eb45-4c2b-add1-d199586065c0\" (UID: \"579ade75-eb45-4c2b-add1-d199586065c0\") "
Oct 30 13:07:11.360451 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.360232    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.360451 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.360256    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.360652 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.358617    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-workload-certs" pod "579ade75-eb45-4c2b-add1-d199586065c0" (UID: "579ade75-eb45-4c2b-add1-d199586065c0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:11.360652 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.358833    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-credential-socket" pod "579ade75-eb45-4c2b-add1-d199586065c0" (UID: "579ade75-eb45-4c2b-add1-d199586065c0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:11.360805 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.359117    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/facb034f-f30c-4290-8ad3-07b58b7163e0-istiod-ca-cert" pod "facb034f-f30c-4290-8ad3-07b58b7163e0" (UID: "facb034f-f30c-4290-8ad3-07b58b7163e0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:07:11.360869 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.360844    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-workload-certs" pod "facb034f-f30c-4290-8ad3-07b58b7163e0" (UID: "facb034f-f30c-4290-8ad3-07b58b7163e0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:11.361037 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.361015    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-workload-socket" pod "facb034f-f30c-4290-8ad3-07b58b7163e0" (UID: "facb034f-f30c-4290-8ad3-07b58b7163e0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:11.361179 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.361160    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/579ade75-eb45-4c2b-add1-d199586065c0-istiod-ca-cert" pod "579ade75-eb45-4c2b-add1-d199586065c0" (UID: "579ade75-eb45-4c2b-add1-d199586065c0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:07:11.361732 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.361696    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-istio-data" pod "579ade75-eb45-4c2b-add1-d199586065c0" (UID: "579ade75-eb45-4c2b-add1-d199586065c0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:11.365723 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.365696    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/579ade75-eb45-4c2b-add1-d199586065c0-istio-token" pod "579ade75-eb45-4c2b-add1-d199586065c0" (UID: "579ade75-eb45-4c2b-add1-d199586065c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:11.365823 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.365770    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-workload-socket" pod "579ade75-eb45-4c2b-add1-d199586065c0" (UID: "579ade75-eb45-4c2b-add1-d199586065c0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:11.371978 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.371753    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-istio-envoy" pod "579ade75-eb45-4c2b-add1-d199586065c0" (UID: "579ade75-eb45-4c2b-add1-d199586065c0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:11.371978 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.371776    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/579ade75-eb45-4c2b-add1-d199586065c0-kube-api-access-j45kf" pod "579ade75-eb45-4c2b-add1-d199586065c0" (UID: "579ade75-eb45-4c2b-add1-d199586065c0"). InnerVolumeSpecName "kube-api-access-j45kf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:11.371978 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.371834    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-envoy" pod "facb034f-f30c-4290-8ad3-07b58b7163e0" (UID: "facb034f-f30c-4290-8ad3-07b58b7163e0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:11.371978 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.371856    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/579ade75-eb45-4c2b-add1-d199586065c0-istio-podinfo" pod "579ade75-eb45-4c2b-add1-d199586065c0" (UID: "579ade75-eb45-4c2b-add1-d199586065c0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:07:11.371978 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.371912    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/facb034f-f30c-4290-8ad3-07b58b7163e0-kube-api-access-f9r8z" pod "facb034f-f30c-4290-8ad3-07b58b7163e0" (UID: "facb034f-f30c-4290-8ad3-07b58b7163e0"). InnerVolumeSpecName "kube-api-access-f9r8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:11.401831 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.381746    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-podinfo" pod "facb034f-f30c-4290-8ad3-07b58b7163e0" (UID: "facb034f-f30c-4290-8ad3-07b58b7163e0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:07:11.401831 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.395708    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-token" pod "facb034f-f30c-4290-8ad3-07b58b7163e0" (UID: "facb034f-f30c-4290-8ad3-07b58b7163e0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:11.463988 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.463864    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.463988 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.463901    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/facb034f-f30c-4290-8ad3-07b58b7163e0-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.464330 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.463914    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.464493 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.464447    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9r8z\" (UniqueName: \"kubernetes.io/projected/facb034f-f30c-4290-8ad3-07b58b7163e0-kube-api-access-f9r8z\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.464816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.464618    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.464816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.464641    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.464816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.464666    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/579ade75-eb45-4c2b-add1-d199586065c0-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.464816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.464680    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/579ade75-eb45-4c2b-add1-d199586065c0-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.464816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.464693    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.464816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.464707    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.464816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.464721    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/facb034f-f30c-4290-8ad3-07b58b7163e0-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.464816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.464734    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.464816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.464748    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/579ade75-eb45-4c2b-add1-d199586065c0-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.464816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.464760    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/facb034f-f30c-4290-8ad3-07b58b7163e0-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.464816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.464774    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/579ade75-eb45-4c2b-add1-d199586065c0-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.464816 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.464787    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j45kf\" (UniqueName: \"kubernetes.io/projected/579ade75-eb45-4c2b-add1-d199586065c0-kube-api-access-j45kf\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.553719 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.553677    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="120ca3bc-b398-43fa-b2d1-1dfa81002f9c" path="/var/lib/kubelet/pods/120ca3bc-b398-43fa-b2d1-1dfa81002f9c/volumes"
Oct 30 13:07:11.568835 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.567738    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 13:07:11.666507 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.666476    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istiod-ca-cert\") pod \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") "
Oct 30 13:07:11.666692 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.666535    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-kube-api-access-djf9q\" (UniqueName: \"kubernetes.io/projected/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-kube-api-access-djf9q\") pod \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") "
Oct 30 13:07:11.666692 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.666652    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-data\") pod \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") "
Oct 30 13:07:11.666794 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.666692    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-workload-socket\") pod \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") "
Oct 30 13:07:11.666794 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.666774    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-token\" (UniqueName: \"kubernetes.io/projected/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-token\") pod \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") "
Oct 30 13:07:11.666909 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.666815    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-podinfo\") pod \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") "
Oct 30 13:07:11.666962 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.666920    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-envoy\") pod \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") "
Oct 30 13:07:11.667013 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.666966    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-data" pod "a9b9135f-faa3-4b2c-ba00-d381772f4ea7" (UID: "a9b9135f-faa3-4b2c-ba00-d381772f4ea7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:11.667065 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.666989    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-workload-certs\") pod \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") "
Oct 30 13:07:11.667065 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.667022    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istiod-ca-cert" pod "a9b9135f-faa3-4b2c-ba00-d381772f4ea7" (UID: "a9b9135f-faa3-4b2c-ba00-d381772f4ea7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:07:11.667180 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.667064    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-workload-socket" pod "a9b9135f-faa3-4b2c-ba00-d381772f4ea7" (UID: "a9b9135f-faa3-4b2c-ba00-d381772f4ea7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:11.667180 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.667077    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-credential-socket\") pod \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\" (UID: \"a9b9135f-faa3-4b2c-ba00-d381772f4ea7\") "
Oct 30 13:07:11.667274 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.667206    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-workload-certs" pod "a9b9135f-faa3-4b2c-ba00-d381772f4ea7" (UID: "a9b9135f-faa3-4b2c-ba00-d381772f4ea7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:11.667324 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.667297    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-credential-socket" pod "a9b9135f-faa3-4b2c-ba00-d381772f4ea7" (UID: "a9b9135f-faa3-4b2c-ba00-d381772f4ea7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:11.667739 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.667718    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.667834 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.667746    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.667834 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.667761    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.667834 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.667774    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.667834 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.667785    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.669327 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.669298    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-token" pod "a9b9135f-faa3-4b2c-ba00-d381772f4ea7" (UID: "a9b9135f-faa3-4b2c-ba00-d381772f4ea7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:11.669447 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.669418    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-kube-api-access-djf9q" pod "a9b9135f-faa3-4b2c-ba00-d381772f4ea7" (UID: "a9b9135f-faa3-4b2c-ba00-d381772f4ea7"). InnerVolumeSpecName "kube-api-access-djf9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:11.669594 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.669530    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-podinfo" pod "a9b9135f-faa3-4b2c-ba00-d381772f4ea7" (UID: "a9b9135f-faa3-4b2c-ba00-d381772f4ea7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:07:11.669709 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.669661    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-envoy" pod "a9b9135f-faa3-4b2c-ba00-d381772f4ea7" (UID: "a9b9135f-faa3-4b2c-ba00-d381772f4ea7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:11.768889 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.768807    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djf9q\" (UniqueName: \"kubernetes.io/projected/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-kube-api-access-djf9q\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.768889 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.768841    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.768889 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.768852    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.768889 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.768859    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9b9135f-faa3-4b2c-ba00-d381772f4ea7-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:11.962715 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.962653    4945 generic.go:356] "Generic (PLEG): container finished" podID="facb034f-f30c-4290-8ad3-07b58b7163e0" containerID="ce68dc9dab1c95a224b9fb561d51f0aa8dbe3f3ea2f94af8b8e1c1612a09874e" exitCode=0
Oct 30 13:07:11.962918 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.962742    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"
Oct 30 13:07:11.962993 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.962742    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr" event={"ID":"facb034f-f30c-4290-8ad3-07b58b7163e0","Type":"ContainerDied","Data":"ce68dc9dab1c95a224b9fb561d51f0aa8dbe3f3ea2f94af8b8e1c1612a09874e"}
Oct 30 13:07:11.963033 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.963005    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr" event={"ID":"facb034f-f30c-4290-8ad3-07b58b7163e0","Type":"ContainerDied","Data":"783821871c6546f6f4205ae0df51ae0b1aed51283a2a936aa9ec5953ce88d94e"}
Oct 30 13:07:11.963086 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.963046    4945 scope.go:122] "RemoveContainer" containerID="ce68dc9dab1c95a224b9fb561d51f0aa8dbe3f3ea2f94af8b8e1c1612a09874e"
Oct 30 13:07:11.965100 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.965074    4945 generic.go:356] "Generic (PLEG): container finished" podID="579ade75-eb45-4c2b-add1-d199586065c0" containerID="595cf7c70eabfad4fbf8790bb1bf32c8f4c9008fb8c061dcfd5aa58281924055" exitCode=0
Oct 30 13:07:11.965716 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.965177    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt" event={"ID":"579ade75-eb45-4c2b-add1-d199586065c0","Type":"ContainerDied","Data":"595cf7c70eabfad4fbf8790bb1bf32c8f4c9008fb8c061dcfd5aa58281924055"}
Oct 30 13:07:11.965716 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.965221    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt" event={"ID":"579ade75-eb45-4c2b-add1-d199586065c0","Type":"ContainerDied","Data":"1539c76052be20aeae1bedccaa5720bd1051b32034ddfbaeb7a7ca428d2da3a9"}
Oct 30 13:07:11.965716 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.965221    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"
Oct 30 13:07:11.967222 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.967196    4945 generic.go:356] "Generic (PLEG): container finished" podID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" containerID="0149777bc760a9584c3f6ec950064b7e65f3cf1829d6af7b991f5f5d2b2c780f" exitCode=0
Oct 30 13:07:11.967346 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.967292    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" event={"ID":"a9b9135f-faa3-4b2c-ba00-d381772f4ea7","Type":"ContainerDied","Data":"0149777bc760a9584c3f6ec950064b7e65f3cf1829d6af7b991f5f5d2b2c780f"}
Oct 30 13:07:11.967346 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.967333    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6" event={"ID":"a9b9135f-faa3-4b2c-ba00-d381772f4ea7","Type":"ContainerDied","Data":"0448043a9358f468297aa947723013170e789c0d961d42a21a67a276cad33761"}
Oct 30 13:07:11.967483 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.967430    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"
Oct 30 13:07:11.975105 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.975084    4945 scope.go:122] "RemoveContainer" containerID="ce68dc9dab1c95a224b9fb561d51f0aa8dbe3f3ea2f94af8b8e1c1612a09874e"
Oct 30 13:07:11.975689 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:07:11.975662    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce68dc9dab1c95a224b9fb561d51f0aa8dbe3f3ea2f94af8b8e1c1612a09874e\": container with ID starting with ce68dc9dab1c95a224b9fb561d51f0aa8dbe3f3ea2f94af8b8e1c1612a09874e not found: ID does not exist" containerID="ce68dc9dab1c95a224b9fb561d51f0aa8dbe3f3ea2f94af8b8e1c1612a09874e"
Oct 30 13:07:11.975835 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.975804    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce68dc9dab1c95a224b9fb561d51f0aa8dbe3f3ea2f94af8b8e1c1612a09874e"} err="failed to get container status \"ce68dc9dab1c95a224b9fb561d51f0aa8dbe3f3ea2f94af8b8e1c1612a09874e\": rpc error: code = NotFound desc = could not find container \"ce68dc9dab1c95a224b9fb561d51f0aa8dbe3f3ea2f94af8b8e1c1612a09874e\": container with ID starting with ce68dc9dab1c95a224b9fb561d51f0aa8dbe3f3ea2f94af8b8e1c1612a09874e not found: ID does not exist"
Oct 30 13:07:11.975932 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.975836    4945 scope.go:122] "RemoveContainer" containerID="595cf7c70eabfad4fbf8790bb1bf32c8f4c9008fb8c061dcfd5aa58281924055"
Oct 30 13:07:11.989225 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.989207    4945 scope.go:122] "RemoveContainer" containerID="595cf7c70eabfad4fbf8790bb1bf32c8f4c9008fb8c061dcfd5aa58281924055"
Oct 30 13:07:11.989567 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:07:11.989541    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"595cf7c70eabfad4fbf8790bb1bf32c8f4c9008fb8c061dcfd5aa58281924055\": container with ID starting with 595cf7c70eabfad4fbf8790bb1bf32c8f4c9008fb8c061dcfd5aa58281924055 not found: ID does not exist" containerID="595cf7c70eabfad4fbf8790bb1bf32c8f4c9008fb8c061dcfd5aa58281924055"
Oct 30 13:07:11.989666 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.989579    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"595cf7c70eabfad4fbf8790bb1bf32c8f4c9008fb8c061dcfd5aa58281924055"} err="failed to get container status \"595cf7c70eabfad4fbf8790bb1bf32c8f4c9008fb8c061dcfd5aa58281924055\": rpc error: code = NotFound desc = could not find container \"595cf7c70eabfad4fbf8790bb1bf32c8f4c9008fb8c061dcfd5aa58281924055\": container with ID starting with 595cf7c70eabfad4fbf8790bb1bf32c8f4c9008fb8c061dcfd5aa58281924055 not found: ID does not exist"
Oct 30 13:07:11.989666 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:11.989627    4945 scope.go:122] "RemoveContainer" containerID="0149777bc760a9584c3f6ec950064b7e65f3cf1829d6af7b991f5f5d2b2c780f"
Oct 30 13:07:12.003375 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:12.003352    4945 scope.go:122] "RemoveContainer" containerID="0149777bc760a9584c3f6ec950064b7e65f3cf1829d6af7b991f5f5d2b2c780f"
Oct 30 13:07:12.003724 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:07:12.003683    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0149777bc760a9584c3f6ec950064b7e65f3cf1829d6af7b991f5f5d2b2c780f\": container with ID starting with 0149777bc760a9584c3f6ec950064b7e65f3cf1829d6af7b991f5f5d2b2c780f not found: ID does not exist" containerID="0149777bc760a9584c3f6ec950064b7e65f3cf1829d6af7b991f5f5d2b2c780f"
Oct 30 13:07:12.003724 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:12.003712    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0149777bc760a9584c3f6ec950064b7e65f3cf1829d6af7b991f5f5d2b2c780f"} err="failed to get container status \"0149777bc760a9584c3f6ec950064b7e65f3cf1829d6af7b991f5f5d2b2c780f\": rpc error: code = NotFound desc = could not find container \"0149777bc760a9584c3f6ec950064b7e65f3cf1829d6af7b991f5f5d2b2c780f\": container with ID starting with 0149777bc760a9584c3f6ec950064b7e65f3cf1829d6af7b991f5f5d2b2c780f not found: ID does not exist"
Oct 30 13:07:12.018662 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:12.018634    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"]
Oct 30 13:07:12.028253 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:12.028193    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-7j5mr"]
Oct 30 13:07:12.059745 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:12.059246    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"]
Oct 30 13:07:12.062394 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:12.062374    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-5sqzt"]
Oct 30 13:07:12.118256 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:12.118223    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"]
Oct 30 13:07:12.151941 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:12.151903    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-c81f42-openshift-default-c7f9d447-2fsz6"]
Oct 30 13:07:13.550903 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:13.550871    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="579ade75-eb45-4c2b-add1-d199586065c0" path="/var/lib/kubelet/pods/579ade75-eb45-4c2b-add1-d199586065c0/volumes"
Oct 30 13:07:13.551405 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:13.551389    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" path="/var/lib/kubelet/pods/a9b9135f-faa3-4b2c-ba00-d381772f4ea7/volumes"
Oct 30 13:07:13.551853 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:13.551839    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="facb034f-f30c-4290-8ad3-07b58b7163e0" path="/var/lib/kubelet/pods/facb034f-f30c-4290-8ad3-07b58b7163e0/volumes"
Oct 30 13:07:18.315466 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:18.315436    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"]
Oct 30 13:07:18.316339 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:18.316306    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" containerName="istio-proxy" containerID="cri-o://52858f0b5a606ad18beff2bfc7ee74a728a5d9dbd4804574c8e480ffe1e9d9d3" gracePeriod=30
Oct 30 13:07:18.320251 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:18.320225    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"]
Oct 30 13:07:18.320509 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:18.320434    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy" containerID="cri-o://48afee0b128284e6b675ee6a12e24b315a85bba4f4108e3c8c28f1634c2c553f" gracePeriod=30
Oct 30 13:07:18.330862 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:18.330833    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"]
Oct 30 13:07:18.332151 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:18.332125    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy" containerID="cri-o://ad3c4e5a8636a349179dd069c1ab44c58110fba3a489619edb9dfde8ee613268" gracePeriod=30
Oct 30 13:07:22.247156 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247117    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"]
Oct 30 13:07:22.247789 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247766    4945 cpu_manager.go:414] "containerMap: removing container" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main"
Oct 30 13:07:22.247789 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247786    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247804    4945 cpu_manager.go:414] "containerMap: removing container" podUID="579ade75-eb45-4c2b-add1-d199586065c0" containerName="istio-proxy"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247810    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="579ade75-eb45-4c2b-add1-d199586065c0" containerName="istio-proxy"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247826    4945 cpu_manager.go:414] "containerMap: removing container" podUID="120ca3bc-b398-43fa-b2d1-1dfa81002f9c" containerName="maas-api"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247832    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="120ca3bc-b398-43fa-b2d1-1dfa81002f9c" containerName="maas-api"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247839    4945 cpu_manager.go:414] "containerMap: removing container" podUID="4ef858d7-259b-4545-ba9f-8e1dd82b8cfd" containerName="maas-api"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247844    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ef858d7-259b-4545-ba9f-8e1dd82b8cfd" containerName="maas-api"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247851    4945 cpu_manager.go:414] "containerMap: removing container" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247855    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247861    4945 cpu_manager.go:414] "containerMap: removing container" podUID="facb034f-f30c-4290-8ad3-07b58b7163e0" containerName="istio-proxy"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247866    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="facb034f-f30c-4290-8ad3-07b58b7163e0" containerName="istio-proxy"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247872    4945 cpu_manager.go:414] "containerMap: removing container" podUID="a069f35f-76b4-4583-857a-304896b3f33d" containerName="istio-proxy"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247876    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a069f35f-76b4-4583-857a-304896b3f33d" containerName="istio-proxy"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247883    4945 cpu_manager.go:414] "containerMap: removing container" podUID="55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e" containerName="maas-api"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247888    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e" containerName="maas-api"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247899    4945 cpu_manager.go:414] "containerMap: removing container" podUID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" containerName="istio-proxy"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247904    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" containerName="istio-proxy"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247916    4945 cpu_manager.go:414] "containerMap: removing container" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerName="istio-proxy"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247921    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerName="istio-proxy"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247926    4945 cpu_manager.go:414] "containerMap: removing container" podUID="81e50924-941f-4d0e-a9a3-c7264a448254" containerName="maas-api"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247931    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81e50924-941f-4d0e-a9a3-c7264a448254" containerName="maas-api"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247942    4945 cpu_manager.go:414] "containerMap: removing container" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy"
Oct 30 13:07:22.247969 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.247948    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy"
Oct 30 13:07:22.249908 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.248057    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="81b43fdc-0af8-4443-a1ba-953d6ad604e4" containerName="istio-proxy"
Oct 30 13:07:22.249908 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.248073    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="81e50924-941f-4d0e-a9a3-c7264a448254" containerName="maas-api"
Oct 30 13:07:22.249908 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.248084    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="026e8b6e-3faa-4835-9815-489c17d71b82" containerName="istio-proxy"
Oct 30 13:07:22.249908 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.248097    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="b41d6c6e-7427-499e-b4b5-b69590e5a458" containerName="main"
Oct 30 13:07:22.249908 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.248106    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="579ade75-eb45-4c2b-add1-d199586065c0" containerName="istio-proxy"
Oct 30 13:07:22.249908 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.248118    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="120ca3bc-b398-43fa-b2d1-1dfa81002f9c" containerName="maas-api"
Oct 30 13:07:22.249908 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.248127    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ef858d7-259b-4545-ba9f-8e1dd82b8cfd" containerName="maas-api"
Oct 30 13:07:22.249908 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.248136    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="819b89b0-600a-4bb8-8a5c-adaa0d941c7e" containerName="main"
Oct 30 13:07:22.249908 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.248144    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="facb034f-f30c-4290-8ad3-07b58b7163e0" containerName="istio-proxy"
Oct 30 13:07:22.249908 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.248153    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="a069f35f-76b4-4583-857a-304896b3f33d" containerName="istio-proxy"
Oct 30 13:07:22.249908 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.248160    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="55dfa7c7-aac9-48cb-b3ac-950b0e3a8c6e" containerName="maas-api"
Oct 30 13:07:22.249908 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.248171    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9b9135f-faa3-4b2c-ba00-d381772f4ea7" containerName="istio-proxy"
Oct 30 13:07:22.252500 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.252473    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.272577 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.272551    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-4cb0fe-openshift-default-dockercfg-4ftfs\""
Oct 30 13:07:22.284205 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.284176    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"]
Oct 30 13:07:22.399039 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.399006    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-token\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.399236 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.399050    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-workload-socket\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.399236 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.399164    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-envoy\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.399236 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.399193    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-credential-socket\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.399389 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.399236    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-data\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.399389 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.399331    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpvkz\" (UniqueName: \"kubernetes.io/projected/042f8c8d-a857-4efb-b8b5-6681e172fe1e-kube-api-access-lpvkz\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.399389 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.399373    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-workload-certs\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.399505 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.399486    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-podinfo\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.399551 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.399516    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istiod-ca-cert\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.500760 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.500693    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-token\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.500760 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.500727    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-workload-socket\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.500994 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.500776    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-envoy\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.500994 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.500805    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-credential-socket\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.500994 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.500847    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-data\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.500994 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.500958    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpvkz\" (UniqueName: \"kubernetes.io/projected/042f8c8d-a857-4efb-b8b5-6681e172fe1e-kube-api-access-lpvkz\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.501289 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.501006    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-workload-certs\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.501289 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.501265    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-podinfo\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.501393 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.501294    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istiod-ca-cert\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.501441 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.501410    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-workload-certs\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.501694 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.501671    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-workload-socket\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.501881 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.501856    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-credential-socket\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.501994 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.501974    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-data\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.502149 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.502130    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istiod-ca-cert\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.503556 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.503538    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-envoy\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.503785 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.503767    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-podinfo\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.520046 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.520026    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-token\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.520556 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.520533    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpvkz\" (UniqueName: \"kubernetes.io/projected/042f8c8d-a857-4efb-b8b5-6681e172fe1e-kube-api-access-lpvkz\") pod \"e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") " pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.566614 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.566584    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:22.883061 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.883029    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"]
Oct 30 13:07:22.886056 ip-10-0-142-36 kubenswrapper[4945]: W1030 13:07:22.886027    4945 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod042f8c8d_a857_4efb_b8b5_6681e172fe1e.slice/crio-176fc3933bc6e52810938b832158fa9d2c1d5ba079e0d7f616b096d9628115e9 WatchSource:0}: Error finding container 176fc3933bc6e52810938b832158fa9d2c1d5ba079e0d7f616b096d9628115e9: Status 404 returned error can't find the container with id 176fc3933bc6e52810938b832158fa9d2c1d5ba079e0d7f616b096d9628115e9
Oct 30 13:07:22.888909 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.888863    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:07:22.888991 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.888949    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:07:22.889035 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:22.888989    4945 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 13:07:23.054171 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.054055    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r" event={"ID":"042f8c8d-a857-4efb-b8b5-6681e172fe1e","Type":"ContainerStarted","Data":"381e1d22e2884c7b5830f65d5cc557c1d2b674d9945cb3739e05410387268828"}
Oct 30 13:07:23.054502 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.054456    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r" event={"ID":"042f8c8d-a857-4efb-b8b5-6681e172fe1e","Type":"ContainerStarted","Data":"176fc3933bc6e52810938b832158fa9d2c1d5ba079e0d7f616b096d9628115e9"}
Oct 30 13:07:23.098120 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.098031    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r" podStartSLOduration=1.098016826 podStartE2EDuration="1.098016826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:07:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:07:23.096473524 +0000 UTC m=+1887.674422612" watchObservedRunningTime="2026-10-30 13:07:23.098016826 +0000 UTC m=+1887.675965915"
Oct 30 13:07:23.571846 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.571774    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:23.594660 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.593691    4945 patch_prober.go:29] interesting pod/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 13:07:23.594660 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.593748    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r" podUID="042f8c8d-a857-4efb-b8b5-6681e172fe1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 13:07:23.698550 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.698510    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 13:07:23.776402 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.776375    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 13:07:23.826112 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.826069    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-istio-data\") pod \"4fb24330-4337-4c41-80d0-e780d94119a4\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") "
Oct 30 13:07:23.826379 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.826355    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4fb24330-4337-4c41-80d0-e780d94119a4-kube-api-access-f64wc\" (UniqueName: \"kubernetes.io/projected/4fb24330-4337-4c41-80d0-e780d94119a4-kube-api-access-f64wc\") pod \"4fb24330-4337-4c41-80d0-e780d94119a4\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") "
Oct 30 13:07:23.826655 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.826591    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-credential-socket\") pod \"4fb24330-4337-4c41-80d0-e780d94119a4\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") "
Oct 30 13:07:23.826809 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.826794    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4fb24330-4337-4c41-80d0-e780d94119a4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fb24330-4337-4c41-80d0-e780d94119a4-istiod-ca-cert\") pod \"4fb24330-4337-4c41-80d0-e780d94119a4\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") "
Oct 30 13:07:23.826954 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.826936    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4fb24330-4337-4c41-80d0-e780d94119a4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fb24330-4337-4c41-80d0-e780d94119a4-istio-podinfo\") pod \"4fb24330-4337-4c41-80d0-e780d94119a4\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") "
Oct 30 13:07:23.827107 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.827083    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-istio-envoy\") pod \"4fb24330-4337-4c41-80d0-e780d94119a4\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") "
Oct 30 13:07:23.827240 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.827121    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-workload-certs\") pod \"4fb24330-4337-4c41-80d0-e780d94119a4\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") "
Oct 30 13:07:23.827240 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.827233    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-workload-socket\") pod \"4fb24330-4337-4c41-80d0-e780d94119a4\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") "
Oct 30 13:07:23.827355 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.827270    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4fb24330-4337-4c41-80d0-e780d94119a4-istio-token\" (UniqueName: \"kubernetes.io/projected/4fb24330-4337-4c41-80d0-e780d94119a4-istio-token\") pod \"4fb24330-4337-4c41-80d0-e780d94119a4\" (UID: \"4fb24330-4337-4c41-80d0-e780d94119a4\") "
Oct 30 13:07:23.828761 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.828716    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-credential-socket" pod "4fb24330-4337-4c41-80d0-e780d94119a4" (UID: "4fb24330-4337-4c41-80d0-e780d94119a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:23.831286 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.831087    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-workload-certs" pod "4fb24330-4337-4c41-80d0-e780d94119a4" (UID: "4fb24330-4337-4c41-80d0-e780d94119a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:23.831286 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.831187    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fb24330-4337-4c41-80d0-e780d94119a4-istio-token" pod "4fb24330-4337-4c41-80d0-e780d94119a4" (UID: "4fb24330-4337-4c41-80d0-e780d94119a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:23.831464 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.831395    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-istio-data" pod "4fb24330-4337-4c41-80d0-e780d94119a4" (UID: "4fb24330-4337-4c41-80d0-e780d94119a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:23.831752 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.831700    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fb24330-4337-4c41-80d0-e780d94119a4-istiod-ca-cert" pod "4fb24330-4337-4c41-80d0-e780d94119a4" (UID: "4fb24330-4337-4c41-80d0-e780d94119a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:07:23.831938 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.831901    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-workload-socket" pod "4fb24330-4337-4c41-80d0-e780d94119a4" (UID: "4fb24330-4337-4c41-80d0-e780d94119a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:23.835250 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.835228    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 13:07:23.835390 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.835365    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fb24330-4337-4c41-80d0-e780d94119a4-kube-api-access-f64wc" pod "4fb24330-4337-4c41-80d0-e780d94119a4" (UID: "4fb24330-4337-4c41-80d0-e780d94119a4"). InnerVolumeSpecName "kube-api-access-f64wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:23.835661 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.835624    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-istio-envoy" pod "4fb24330-4337-4c41-80d0-e780d94119a4" (UID: "4fb24330-4337-4c41-80d0-e780d94119a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:23.835757 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.835702    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4fb24330-4337-4c41-80d0-e780d94119a4-istio-podinfo" pod "4fb24330-4337-4c41-80d0-e780d94119a4" (UID: "4fb24330-4337-4c41-80d0-e780d94119a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:07:23.928952 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.928663    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-workload-socket\") pod \"d0ba9b13-4372-4d8f-9916-30246cc34e84\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") "
Oct 30 13:07:23.928952 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.928724    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-workload-certs\") pod \"d0ba9b13-4372-4d8f-9916-30246cc34e84\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") "
Oct 30 13:07:23.928952 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.928791    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-podinfo\") pod \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") "
Oct 30 13:07:23.928952 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.928846    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-data\") pod \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") "
Oct 30 13:07:23.929286 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929075    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-workload-socket" pod "d0ba9b13-4372-4d8f-9916-30246cc34e84" (UID: "d0ba9b13-4372-4d8f-9916-30246cc34e84"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:23.929343 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929294    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-data" pod "f3611f91-0fbc-47ff-b790-d635fb9d2e21" (UID: "f3611f91-0fbc-47ff-b790-d635fb9d2e21"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:23.929343 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929296    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-workload-certs" pod "d0ba9b13-4372-4d8f-9916-30246cc34e84" (UID: "d0ba9b13-4372-4d8f-9916-30246cc34e84"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:23.929508 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929485    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-podinfo\") pod \"d0ba9b13-4372-4d8f-9916-30246cc34e84\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") "
Oct 30 13:07:23.929666 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929514    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-envoy\") pod \"d0ba9b13-4372-4d8f-9916-30246cc34e84\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") "
Oct 30 13:07:23.929666 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929540    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0ba9b13-4372-4d8f-9916-30246cc34e84-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0ba9b13-4372-4d8f-9916-30246cc34e84-istiod-ca-cert\") pod \"d0ba9b13-4372-4d8f-9916-30246cc34e84\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") "
Oct 30 13:07:23.929666 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929571    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-token\" (UniqueName: \"kubernetes.io/projected/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-token\") pod \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") "
Oct 30 13:07:23.929666 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929646    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0ba9b13-4372-4d8f-9916-30246cc34e84-kube-api-access-kl7qz\" (UniqueName: \"kubernetes.io/projected/d0ba9b13-4372-4d8f-9916-30246cc34e84-kube-api-access-kl7qz\") pod \"d0ba9b13-4372-4d8f-9916-30246cc34e84\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") "
Oct 30 13:07:23.929895 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929694    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-data\") pod \"d0ba9b13-4372-4d8f-9916-30246cc34e84\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") "
Oct 30 13:07:23.929895 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929721    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istiod-ca-cert\") pod \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") "
Oct 30 13:07:23.929895 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929762    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-credential-socket\") pod \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") "
Oct 30 13:07:23.929895 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929785    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3611f91-0fbc-47ff-b790-d635fb9d2e21-kube-api-access-bmr2r\" (UniqueName: \"kubernetes.io/projected/f3611f91-0fbc-47ff-b790-d635fb9d2e21-kube-api-access-bmr2r\") pod \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") "
Oct 30 13:07:23.929895 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929812    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-envoy\") pod \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") "
Oct 30 13:07:23.929895 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929848    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-workload-certs\") pod \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") "
Oct 30 13:07:23.929895 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929866    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-credential-socket\") pod \"d0ba9b13-4372-4d8f-9916-30246cc34e84\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") "
Oct 30 13:07:23.929895 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929889    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-workload-socket\") pod \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\" (UID: \"f3611f91-0fbc-47ff-b790-d635fb9d2e21\") "
Oct 30 13:07:23.930377 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.929914    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-token\" (UniqueName: \"kubernetes.io/projected/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-token\") pod \"d0ba9b13-4372-4d8f-9916-30246cc34e84\" (UID: \"d0ba9b13-4372-4d8f-9916-30246cc34e84\") "
Oct 30 13:07:23.930438 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930374    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f64wc\" (UniqueName: \"kubernetes.io/projected/4fb24330-4337-4c41-80d0-e780d94119a4-kube-api-access-f64wc\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:23.930438 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930396    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:23.930438 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930411    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:23.930438 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930427    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4fb24330-4337-4c41-80d0-e780d94119a4-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:23.930660 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930443    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4fb24330-4337-4c41-80d0-e780d94119a4-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:23.930660 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930455    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:23.930660 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930468    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:23.930660 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930480    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:23.930660 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930494    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4fb24330-4337-4c41-80d0-e780d94119a4-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:23.930660 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930508    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:23.930660 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930519    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:23.930660 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930532    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4fb24330-4337-4c41-80d0-e780d94119a4-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:23.930660 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930562    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-credential-socket" pod "d0ba9b13-4372-4d8f-9916-30246cc34e84" (UID: "d0ba9b13-4372-4d8f-9916-30246cc34e84"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:23.930660 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930646    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0ba9b13-4372-4d8f-9916-30246cc34e84-istiod-ca-cert" pod "d0ba9b13-4372-4d8f-9916-30246cc34e84" (UID: "d0ba9b13-4372-4d8f-9916-30246cc34e84"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:07:23.931205 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930753    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-data" pod "d0ba9b13-4372-4d8f-9916-30246cc34e84" (UID: "d0ba9b13-4372-4d8f-9916-30246cc34e84"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:23.931205 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.930813    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-workload-certs" pod "f3611f91-0fbc-47ff-b790-d635fb9d2e21" (UID: "f3611f91-0fbc-47ff-b790-d635fb9d2e21"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:23.931205 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.931011    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-credential-socket" pod "f3611f91-0fbc-47ff-b790-d635fb9d2e21" (UID: "f3611f91-0fbc-47ff-b790-d635fb9d2e21"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:23.931492 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.931471    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istiod-ca-cert" pod "f3611f91-0fbc-47ff-b790-d635fb9d2e21" (UID: "f3611f91-0fbc-47ff-b790-d635fb9d2e21"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:07:23.933635 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.932808    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-podinfo" pod "f3611f91-0fbc-47ff-b790-d635fb9d2e21" (UID: "f3611f91-0fbc-47ff-b790-d635fb9d2e21"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:07:23.933635 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.932961    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-workload-socket" pod "f3611f91-0fbc-47ff-b790-d635fb9d2e21" (UID: "f3611f91-0fbc-47ff-b790-d635fb9d2e21"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:23.933635 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.933034    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-token" pod "d0ba9b13-4372-4d8f-9916-30246cc34e84" (UID: "d0ba9b13-4372-4d8f-9916-30246cc34e84"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:23.933635 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.933473    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-envoy" pod "d0ba9b13-4372-4d8f-9916-30246cc34e84" (UID: "d0ba9b13-4372-4d8f-9916-30246cc34e84"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:23.934483 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.934458    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0ba9b13-4372-4d8f-9916-30246cc34e84-kube-api-access-kl7qz" pod "d0ba9b13-4372-4d8f-9916-30246cc34e84" (UID: "d0ba9b13-4372-4d8f-9916-30246cc34e84"). InnerVolumeSpecName "kube-api-access-kl7qz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:23.934679 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.934653    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-envoy" pod "f3611f91-0fbc-47ff-b790-d635fb9d2e21" (UID: "f3611f91-0fbc-47ff-b790-d635fb9d2e21"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:07:23.934808 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.934785    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-token" pod "f3611f91-0fbc-47ff-b790-d635fb9d2e21" (UID: "f3611f91-0fbc-47ff-b790-d635fb9d2e21"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:23.935064 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.935038    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3611f91-0fbc-47ff-b790-d635fb9d2e21-kube-api-access-bmr2r" pod "f3611f91-0fbc-47ff-b790-d635fb9d2e21" (UID: "f3611f91-0fbc-47ff-b790-d635fb9d2e21"). InnerVolumeSpecName "kube-api-access-bmr2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:07:23.936335 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:23.936316    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-podinfo" pod "d0ba9b13-4372-4d8f-9916-30246cc34e84" (UID: "d0ba9b13-4372-4d8f-9916-30246cc34e84"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:07:24.044528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.039759    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:24.044528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.039813    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmr2r\" (UniqueName: \"kubernetes.io/projected/f3611f91-0fbc-47ff-b790-d635fb9d2e21-kube-api-access-bmr2r\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:24.044528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.039829    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:24.044528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.039851    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:24.044528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.039865    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:24.044528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.039891    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3611f91-0fbc-47ff-b790-d635fb9d2e21-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:24.044528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.039903    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:24.044528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.039921    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:24.044528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.039933    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:24.044528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.039947    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:24.044528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.039959    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0ba9b13-4372-4d8f-9916-30246cc34e84-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:24.044528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.039975    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:24.044528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.039987    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl7qz\" (UniqueName: \"kubernetes.io/projected/d0ba9b13-4372-4d8f-9916-30246cc34e84-kube-api-access-kl7qz\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:24.044528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.039999    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0ba9b13-4372-4d8f-9916-30246cc34e84-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:24.044528 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.040010    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3611f91-0fbc-47ff-b790-d635fb9d2e21-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:07:24.063058 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.063024    4945 generic.go:356] "Generic (PLEG): container finished" podID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerID="ad3c4e5a8636a349179dd069c1ab44c58110fba3a489619edb9dfde8ee613268" exitCode=0
Oct 30 13:07:24.063218 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.063085    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"
Oct 30 13:07:24.063218 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.063104    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" event={"ID":"d0ba9b13-4372-4d8f-9916-30246cc34e84","Type":"ContainerDied","Data":"ad3c4e5a8636a349179dd069c1ab44c58110fba3a489619edb9dfde8ee613268"}
Oct 30 13:07:24.063218 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.063181    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg" event={"ID":"d0ba9b13-4372-4d8f-9916-30246cc34e84","Type":"ContainerDied","Data":"ca6a4928a942a152525ec4c085067dc80369841ece2c47a5a36752cdcf72ab33"}
Oct 30 13:07:24.063218 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.063213    4945 scope.go:122] "RemoveContainer" containerID="ad3c4e5a8636a349179dd069c1ab44c58110fba3a489619edb9dfde8ee613268"
Oct 30 13:07:24.064529 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.064502    4945 generic.go:356] "Generic (PLEG): container finished" podID="4fb24330-4337-4c41-80d0-e780d94119a4" containerID="52858f0b5a606ad18beff2bfc7ee74a728a5d9dbd4804574c8e480ffe1e9d9d3" exitCode=0
Oct 30 13:07:24.064726 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.064565    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"
Oct 30 13:07:24.064726 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.064590    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" event={"ID":"4fb24330-4337-4c41-80d0-e780d94119a4","Type":"ContainerDied","Data":"52858f0b5a606ad18beff2bfc7ee74a728a5d9dbd4804574c8e480ffe1e9d9d3"}
Oct 30 13:07:24.064726 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.064676    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl" event={"ID":"4fb24330-4337-4c41-80d0-e780d94119a4","Type":"ContainerDied","Data":"988f4d75b3fb841c70c10a3b6a0e699018918a1acc47fdb17605ba3d7231ad35"}
Oct 30 13:07:24.066236 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.066210    4945 generic.go:356] "Generic (PLEG): container finished" podID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerID="48afee0b128284e6b675ee6a12e24b315a85bba4f4108e3c8c28f1634c2c553f" exitCode=0
Oct 30 13:07:24.066353 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.066259    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"
Oct 30 13:07:24.066353 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.066268    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" event={"ID":"f3611f91-0fbc-47ff-b790-d635fb9d2e21","Type":"ContainerDied","Data":"48afee0b128284e6b675ee6a12e24b315a85bba4f4108e3c8c28f1634c2c553f"}
Oct 30 13:07:24.066469 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.066350    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj" event={"ID":"f3611f91-0fbc-47ff-b790-d635fb9d2e21","Type":"ContainerDied","Data":"d853c6bb3b95ac0c84ac6a6ca2077ad927c53334b0ffef813a7a05163474f64a"}
Oct 30 13:07:24.127867 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.127837    4945 scope.go:122] "RemoveContainer" containerID="ad3c4e5a8636a349179dd069c1ab44c58110fba3a489619edb9dfde8ee613268"
Oct 30 13:07:24.129702 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:07:24.129647    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad3c4e5a8636a349179dd069c1ab44c58110fba3a489619edb9dfde8ee613268\": container with ID starting with ad3c4e5a8636a349179dd069c1ab44c58110fba3a489619edb9dfde8ee613268 not found: ID does not exist" containerID="ad3c4e5a8636a349179dd069c1ab44c58110fba3a489619edb9dfde8ee613268"
Oct 30 13:07:24.129702 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.129683    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad3c4e5a8636a349179dd069c1ab44c58110fba3a489619edb9dfde8ee613268"} err="failed to get container status \"ad3c4e5a8636a349179dd069c1ab44c58110fba3a489619edb9dfde8ee613268\": rpc error: code = NotFound desc = could not find container \"ad3c4e5a8636a349179dd069c1ab44c58110fba3a489619edb9dfde8ee613268\": container with ID starting with ad3c4e5a8636a349179dd069c1ab44c58110fba3a489619edb9dfde8ee613268 not found: ID does not exist"
Oct 30 13:07:24.129702 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.129702    4945 scope.go:122] "RemoveContainer" containerID="52858f0b5a606ad18beff2bfc7ee74a728a5d9dbd4804574c8e480ffe1e9d9d3"
Oct 30 13:07:24.141043 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.141001    4945 scope.go:122] "RemoveContainer" containerID="52858f0b5a606ad18beff2bfc7ee74a728a5d9dbd4804574c8e480ffe1e9d9d3"
Oct 30 13:07:24.141339 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:07:24.141316    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52858f0b5a606ad18beff2bfc7ee74a728a5d9dbd4804574c8e480ffe1e9d9d3\": container with ID starting with 52858f0b5a606ad18beff2bfc7ee74a728a5d9dbd4804574c8e480ffe1e9d9d3 not found: ID does not exist" containerID="52858f0b5a606ad18beff2bfc7ee74a728a5d9dbd4804574c8e480ffe1e9d9d3"
Oct 30 13:07:24.141409 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.141356    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52858f0b5a606ad18beff2bfc7ee74a728a5d9dbd4804574c8e480ffe1e9d9d3"} err="failed to get container status \"52858f0b5a606ad18beff2bfc7ee74a728a5d9dbd4804574c8e480ffe1e9d9d3\": rpc error: code = NotFound desc = could not find container \"52858f0b5a606ad18beff2bfc7ee74a728a5d9dbd4804574c8e480ffe1e9d9d3\": container with ID starting with 52858f0b5a606ad18beff2bfc7ee74a728a5d9dbd4804574c8e480ffe1e9d9d3 not found: ID does not exist"
Oct 30 13:07:24.141409 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.141372    4945 scope.go:122] "RemoveContainer" containerID="48afee0b128284e6b675ee6a12e24b315a85bba4f4108e3c8c28f1634c2c553f"
Oct 30 13:07:24.150876 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.150856    4945 scope.go:122] "RemoveContainer" containerID="48afee0b128284e6b675ee6a12e24b315a85bba4f4108e3c8c28f1634c2c553f"
Oct 30 13:07:24.151143 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:07:24.151124    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48afee0b128284e6b675ee6a12e24b315a85bba4f4108e3c8c28f1634c2c553f\": container with ID starting with 48afee0b128284e6b675ee6a12e24b315a85bba4f4108e3c8c28f1634c2c553f not found: ID does not exist" containerID="48afee0b128284e6b675ee6a12e24b315a85bba4f4108e3c8c28f1634c2c553f"
Oct 30 13:07:24.151187 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.151150    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48afee0b128284e6b675ee6a12e24b315a85bba4f4108e3c8c28f1634c2c553f"} err="failed to get container status \"48afee0b128284e6b675ee6a12e24b315a85bba4f4108e3c8c28f1634c2c553f\": rpc error: code = NotFound desc = could not find container \"48afee0b128284e6b675ee6a12e24b315a85bba4f4108e3c8c28f1634c2c553f\": container with ID starting with 48afee0b128284e6b675ee6a12e24b315a85bba4f4108e3c8c28f1634c2c553f not found: ID does not exist"
Oct 30 13:07:24.169705 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.169676    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"]
Oct 30 13:07:24.173926 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.173899    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-m9fkg"]
Oct 30 13:07:24.215032 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.214995    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"]
Oct 30 13:07:24.220678 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.220646    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-h9djl"]
Oct 30 13:07:24.252119 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.252092    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"]
Oct 30 13:07:24.256061 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.256035    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-4cd3fb-openshift-default-79cbdb49d5-699gj"]
Oct 30 13:07:24.566918 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.566887    4945 patch_prober.go:29] interesting pod/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 13:07:24.567067 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:24.566937    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r" podUID="042f8c8d-a857-4efb-b8b5-6681e172fe1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 13:07:25.549175 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:25.549142    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" path="/var/lib/kubelet/pods/4fb24330-4337-4c41-80d0-e780d94119a4/volumes"
Oct 30 13:07:25.549671 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:25.549654    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" path="/var/lib/kubelet/pods/d0ba9b13-4372-4d8f-9916-30246cc34e84/volumes"
Oct 30 13:07:25.550082 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:25.550070    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" path="/var/lib/kubelet/pods/f3611f91-0fbc-47ff-b790-d635fb9d2e21/volumes"
Oct 30 13:07:25.566913 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:25.566888    4945 patch_prober.go:29] interesting pod/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 13:07:25.567056 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:25.566933    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r" podUID="042f8c8d-a857-4efb-b8b5-6681e172fe1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 13:07:26.227488 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:07:26.227449    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:07:26.567005 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:26.566916    4945 patch_prober.go:29] interesting pod/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 13:07:26.567005 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:26.566974    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r" podUID="042f8c8d-a857-4efb-b8b5-6681e172fe1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 13:07:27.566861 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:27.566832    4945 patch_prober.go:29] interesting pod/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 13:07:27.567103 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:27.567079    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r" podUID="042f8c8d-a857-4efb-b8b5-6681e172fe1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 13:07:28.567347 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:28.567313    4945 patch_prober.go:29] interesting pod/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 13:07:28.567757 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:28.567381    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r" podUID="042f8c8d-a857-4efb-b8b5-6681e172fe1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 13:07:29.566944 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:29.566907    4945 patch_prober.go:29] interesting pod/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused" start-of-body=
Oct 30 13:07:29.567128 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:29.566968    4945 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r" podUID="042f8c8d-a857-4efb-b8b5-6681e172fe1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.109:15021/healthz/ready\": dial tcp 10.132.0.109:15021: connect: connection refused"
Oct 30 13:07:30.572740 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:30.572699    4945 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:30.573389 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:30.573361    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:30.574253 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:07:30.574227    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:07:56.198940 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:07:56.198896    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:08:22.538839 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:22.538730    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:08:22.568091 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:22.568051    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:08:22.896797 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:22.896688    4945 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:08:26.194914 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:08:26.194875    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:08:36.401132 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:36.400666    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"]
Oct 30 13:08:36.401132 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:36.400920    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd" podUID="579928aa-fcf1-4385-95ac-5d8f8648e5be" containerName="maas-api" containerID="cri-o://82c0014e965898e5f4c5428011689faa605c101bee21af178462e1773ffdbc05" gracePeriod=30
Oct 30 13:08:37.323653 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.323588    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd" podUID="579928aa-fcf1-4385-95ac-5d8f8648e5be" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.96:8443/health\": dial tcp 10.132.0.96:8443: connect: connection refused"
Oct 30 13:08:37.697746 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.697700    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:08:37.751712 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.751668    4945 generic.go:356] "Generic (PLEG): container finished" podID="579928aa-fcf1-4385-95ac-5d8f8648e5be" containerID="82c0014e965898e5f4c5428011689faa605c101bee21af178462e1773ffdbc05" exitCode=0
Oct 30 13:08:37.751873 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.751719    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"
Oct 30 13:08:37.751873 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.751748    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd" event={"ID":"579928aa-fcf1-4385-95ac-5d8f8648e5be","Type":"ContainerDied","Data":"82c0014e965898e5f4c5428011689faa605c101bee21af178462e1773ffdbc05"}
Oct 30 13:08:37.751873 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.751808    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd" event={"ID":"579928aa-fcf1-4385-95ac-5d8f8648e5be","Type":"ContainerDied","Data":"117d999e25479ba9ce6e4c298ecc59ccad36ff801e3ace6fa241f7ca11096733"}
Oct 30 13:08:37.751873 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.751834    4945 scope.go:122] "RemoveContainer" containerID="82c0014e965898e5f4c5428011689faa605c101bee21af178462e1773ffdbc05"
Oct 30 13:08:37.764066 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.764036    4945 scope.go:122] "RemoveContainer" containerID="82c0014e965898e5f4c5428011689faa605c101bee21af178462e1773ffdbc05"
Oct 30 13:08:37.764495 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:08:37.764462    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82c0014e965898e5f4c5428011689faa605c101bee21af178462e1773ffdbc05\": container with ID starting with 82c0014e965898e5f4c5428011689faa605c101bee21af178462e1773ffdbc05 not found: ID does not exist" containerID="82c0014e965898e5f4c5428011689faa605c101bee21af178462e1773ffdbc05"
Oct 30 13:08:37.764624 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.764503    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82c0014e965898e5f4c5428011689faa605c101bee21af178462e1773ffdbc05"} err="failed to get container status \"82c0014e965898e5f4c5428011689faa605c101bee21af178462e1773ffdbc05\": rpc error: code = NotFound desc = could not find container \"82c0014e965898e5f4c5428011689faa605c101bee21af178462e1773ffdbc05\": container with ID starting with 82c0014e965898e5f4c5428011689faa605c101bee21af178462e1773ffdbc05 not found: ID does not exist"
Oct 30 13:08:37.798776 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.798743    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-metrics-certs\" (UniqueName: \"kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-metrics-certs\") pod \"579928aa-fcf1-4385-95ac-5d8f8648e5be\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") "
Oct 30 13:08:37.798942 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.798863    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/579928aa-fcf1-4385-95ac-5d8f8648e5be-kube-api-access-wpnjq\" (UniqueName: \"kubernetes.io/projected/579928aa-fcf1-4385-95ac-5d8f8648e5be-kube-api-access-wpnjq\") pod \"579928aa-fcf1-4385-95ac-5d8f8648e5be\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") "
Oct 30 13:08:37.798942 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.798913    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/579928aa-fcf1-4385-95ac-5d8f8648e5be-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/579928aa-fcf1-4385-95ac-5d8f8648e5be-openshift-service-ca\") pod \"579928aa-fcf1-4385-95ac-5d8f8648e5be\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") "
Oct 30 13:08:37.799067 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.798944    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-maas-api-tls\") pod \"579928aa-fcf1-4385-95ac-5d8f8648e5be\" (UID: \"579928aa-fcf1-4385-95ac-5d8f8648e5be\") "
Oct 30 13:08:37.799363 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.799330    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/579928aa-fcf1-4385-95ac-5d8f8648e5be-openshift-service-ca" pod "579928aa-fcf1-4385-95ac-5d8f8648e5be" (UID: "579928aa-fcf1-4385-95ac-5d8f8648e5be"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:08:37.800880 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.800855    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-metrics-certs" pod "579928aa-fcf1-4385-95ac-5d8f8648e5be" (UID: "579928aa-fcf1-4385-95ac-5d8f8648e5be"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:08:37.801033 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.801016    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-maas-api-tls" pod "579928aa-fcf1-4385-95ac-5d8f8648e5be" (UID: "579928aa-fcf1-4385-95ac-5d8f8648e5be"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:08:37.801382 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.801368    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/579928aa-fcf1-4385-95ac-5d8f8648e5be-kube-api-access-wpnjq" pod "579928aa-fcf1-4385-95ac-5d8f8648e5be" (UID: "579928aa-fcf1-4385-95ac-5d8f8648e5be"). InnerVolumeSpecName "kube-api-access-wpnjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:08:37.900266 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.900185    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/579928aa-fcf1-4385-95ac-5d8f8648e5be-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:08:37.900266 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.900220    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:08:37.900266 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.900236    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/579928aa-fcf1-4385-95ac-5d8f8648e5be-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:08:37.900266 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:37.900249    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpnjq\" (UniqueName: \"kubernetes.io/projected/579928aa-fcf1-4385-95ac-5d8f8648e5be-kube-api-access-wpnjq\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:08:38.097268 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:38.097230    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"]
Oct 30 13:08:38.108080 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:38.108048    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-026c2f-56b67c7bc4-q2xnd"]
Oct 30 13:08:39.549743 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:08:39.549709    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="579928aa-fcf1-4385-95ac-5d8f8648e5be" path="/var/lib/kubelet/pods/579928aa-fcf1-4385-95ac-5d8f8648e5be/volumes"
Oct 30 13:08:56.192153 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:08:56.192113    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:09:16.444253 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:16.444215    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"]
Oct 30 13:09:16.445349 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:16.444467    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8" podUID="951278f9-b9f4-43bd-920c-fb92e3fbed25" containerName="istio-proxy" containerID="cri-o://146062a33cfe6066a21f08340a14679945e26f6d406497d0689f6170d39f02b2" gracePeriod=30
Oct 30 13:09:18.363767 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:18.363730    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"]
Oct 30 13:09:18.364559 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:18.364521    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r" podUID="042f8c8d-a857-4efb-b8b5-6681e172fe1e" containerName="istio-proxy" containerID="cri-o://381e1d22e2884c7b5830f65d5cc557c1d2b674d9945cb3739e05410387268828" gracePeriod=30
Oct 30 13:09:21.719061 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.718985    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 13:09:21.840792 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.840761    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-workload-certs\") pod \"951278f9-b9f4-43bd-920c-fb92e3fbed25\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") "
Oct 30 13:09:21.840973 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.840826    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/951278f9-b9f4-43bd-920c-fb92e3fbed25-kube-api-access-hf485\" (UniqueName: \"kubernetes.io/projected/951278f9-b9f4-43bd-920c-fb92e3fbed25-kube-api-access-hf485\") pod \"951278f9-b9f4-43bd-920c-fb92e3fbed25\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") "
Oct 30 13:09:21.840973 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.840852    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-token\" (UniqueName: \"kubernetes.io/projected/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-token\") pod \"951278f9-b9f4-43bd-920c-fb92e3fbed25\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") "
Oct 30 13:09:21.840973 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.840875    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-credential-socket\") pod \"951278f9-b9f4-43bd-920c-fb92e3fbed25\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") "
Oct 30 13:09:21.840973 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.840921    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-data\") pod \"951278f9-b9f4-43bd-920c-fb92e3fbed25\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") "
Oct 30 13:09:21.840973 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.840943    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-workload-socket\") pod \"951278f9-b9f4-43bd-920c-fb92e3fbed25\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") "
Oct 30 13:09:21.841294 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.841007    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/951278f9-b9f4-43bd-920c-fb92e3fbed25-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/951278f9-b9f4-43bd-920c-fb92e3fbed25-istiod-ca-cert\") pod \"951278f9-b9f4-43bd-920c-fb92e3fbed25\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") "
Oct 30 13:09:21.841294 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.841082    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-envoy\") pod \"951278f9-b9f4-43bd-920c-fb92e3fbed25\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") "
Oct 30 13:09:21.841294 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.841104    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-workload-certs" pod "951278f9-b9f4-43bd-920c-fb92e3fbed25" (UID: "951278f9-b9f4-43bd-920c-fb92e3fbed25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:09:21.841294 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.841129    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-podinfo\") pod \"951278f9-b9f4-43bd-920c-fb92e3fbed25\" (UID: \"951278f9-b9f4-43bd-920c-fb92e3fbed25\") "
Oct 30 13:09:21.841294 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.841239    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-credential-socket" pod "951278f9-b9f4-43bd-920c-fb92e3fbed25" (UID: "951278f9-b9f4-43bd-920c-fb92e3fbed25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:09:21.841552 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.841393    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-workload-socket" pod "951278f9-b9f4-43bd-920c-fb92e3fbed25" (UID: "951278f9-b9f4-43bd-920c-fb92e3fbed25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:09:21.841552 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.841445    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/951278f9-b9f4-43bd-920c-fb92e3fbed25-istiod-ca-cert" pod "951278f9-b9f4-43bd-920c-fb92e3fbed25" (UID: "951278f9-b9f4-43bd-920c-fb92e3fbed25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:09:21.841696 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.841572    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-data" pod "951278f9-b9f4-43bd-920c-fb92e3fbed25" (UID: "951278f9-b9f4-43bd-920c-fb92e3fbed25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:09:21.841844 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.841821    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:21.841935 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.841848    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:21.841935 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.841863    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:21.841935 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.841877    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:21.841935 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.841890    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/951278f9-b9f4-43bd-920c-fb92e3fbed25-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:21.843325 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.843292    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-token" pod "951278f9-b9f4-43bd-920c-fb92e3fbed25" (UID: "951278f9-b9f4-43bd-920c-fb92e3fbed25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:09:21.843439 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.843394    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-envoy" pod "951278f9-b9f4-43bd-920c-fb92e3fbed25" (UID: "951278f9-b9f4-43bd-920c-fb92e3fbed25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:09:21.843439 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.843415    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-podinfo" pod "951278f9-b9f4-43bd-920c-fb92e3fbed25" (UID: "951278f9-b9f4-43bd-920c-fb92e3fbed25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:09:21.843834 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.843807    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/951278f9-b9f4-43bd-920c-fb92e3fbed25-kube-api-access-hf485" pod "951278f9-b9f4-43bd-920c-fb92e3fbed25" (UID: "951278f9-b9f4-43bd-920c-fb92e3fbed25"). InnerVolumeSpecName "kube-api-access-hf485". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:09:21.943203 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.943165    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:21.943203 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.943203    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:21.943484 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.943219    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hf485\" (UniqueName: \"kubernetes.io/projected/951278f9-b9f4-43bd-920c-fb92e3fbed25-kube-api-access-hf485\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:21.943484 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:21.943233    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/951278f9-b9f4-43bd-920c-fb92e3fbed25-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:22.337986 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:22.337944    4945 generic.go:356] "Generic (PLEG): container finished" podID="951278f9-b9f4-43bd-920c-fb92e3fbed25" containerID="146062a33cfe6066a21f08340a14679945e26f6d406497d0689f6170d39f02b2" exitCode=0
Oct 30 13:09:22.338163 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:22.338015    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"
Oct 30 13:09:22.338163 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:22.338028    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8" event={"ID":"951278f9-b9f4-43bd-920c-fb92e3fbed25","Type":"ContainerDied","Data":"146062a33cfe6066a21f08340a14679945e26f6d406497d0689f6170d39f02b2"}
Oct 30 13:09:22.338163 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:22.338114    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8" event={"ID":"951278f9-b9f4-43bd-920c-fb92e3fbed25","Type":"ContainerDied","Data":"e87ed63ba0b9a3766bc7383658b76565d5147fd3a839fa8024e4a48de2a05676"}
Oct 30 13:09:22.338163 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:22.338138    4945 scope.go:122] "RemoveContainer" containerID="146062a33cfe6066a21f08340a14679945e26f6d406497d0689f6170d39f02b2"
Oct 30 13:09:22.350801 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:22.350778    4945 scope.go:122] "RemoveContainer" containerID="146062a33cfe6066a21f08340a14679945e26f6d406497d0689f6170d39f02b2"
Oct 30 13:09:22.351073 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:09:22.351047    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"146062a33cfe6066a21f08340a14679945e26f6d406497d0689f6170d39f02b2\": container with ID starting with 146062a33cfe6066a21f08340a14679945e26f6d406497d0689f6170d39f02b2 not found: ID does not exist" containerID="146062a33cfe6066a21f08340a14679945e26f6d406497d0689f6170d39f02b2"
Oct 30 13:09:22.351143 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:22.351085    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"146062a33cfe6066a21f08340a14679945e26f6d406497d0689f6170d39f02b2"} err="failed to get container status \"146062a33cfe6066a21f08340a14679945e26f6d406497d0689f6170d39f02b2\": rpc error: code = NotFound desc = could not find container \"146062a33cfe6066a21f08340a14679945e26f6d406497d0689f6170d39f02b2\": container with ID starting with 146062a33cfe6066a21f08340a14679945e26f6d406497d0689f6170d39f02b2 not found: ID does not exist"
Oct 30 13:09:22.388402 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:22.388348    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"]
Oct 30 13:09:22.400025 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:22.400000    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-f74x8"]
Oct 30 13:09:23.554728 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.554693    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="951278f9-b9f4-43bd-920c-fb92e3fbed25" path="/var/lib/kubelet/pods/951278f9-b9f4-43bd-920c-fb92e3fbed25/volumes"
Oct 30 13:09:23.647982 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.647957    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:09:23.764187 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.764157    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-envoy\") pod \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") "
Oct 30 13:09:23.764365 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.764194    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-workload-socket\") pod \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") "
Oct 30 13:09:23.764365 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.764241    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-token\" (UniqueName: \"kubernetes.io/projected/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-token\") pod \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") "
Oct 30 13:09:23.764365 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.764289    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-workload-certs\") pod \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") "
Oct 30 13:09:23.764365 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.764312    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-podinfo\") pod \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") "
Oct 30 13:09:23.764365 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.764340    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istiod-ca-cert\") pod \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") "
Oct 30 13:09:23.764365 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.764365    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/042f8c8d-a857-4efb-b8b5-6681e172fe1e-kube-api-access-lpvkz\" (UniqueName: \"kubernetes.io/projected/042f8c8d-a857-4efb-b8b5-6681e172fe1e-kube-api-access-lpvkz\") pod \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") "
Oct 30 13:09:23.764708 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.764474    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-data\") pod \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") "
Oct 30 13:09:23.764708 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.764547    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-workload-certs" pod "042f8c8d-a857-4efb-b8b5-6681e172fe1e" (UID: "042f8c8d-a857-4efb-b8b5-6681e172fe1e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:09:23.764708 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.764562    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-credential-socket\") pod \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\" (UID: \"042f8c8d-a857-4efb-b8b5-6681e172fe1e\") "
Oct 30 13:09:23.764940 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.764692    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-workload-socket" pod "042f8c8d-a857-4efb-b8b5-6681e172fe1e" (UID: "042f8c8d-a857-4efb-b8b5-6681e172fe1e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:09:23.764940 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.764806    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-credential-socket" pod "042f8c8d-a857-4efb-b8b5-6681e172fe1e" (UID: "042f8c8d-a857-4efb-b8b5-6681e172fe1e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:09:23.765060 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.764979    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-data" pod "042f8c8d-a857-4efb-b8b5-6681e172fe1e" (UID: "042f8c8d-a857-4efb-b8b5-6681e172fe1e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:09:23.765060 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.764906    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istiod-ca-cert" pod "042f8c8d-a857-4efb-b8b5-6681e172fe1e" (UID: "042f8c8d-a857-4efb-b8b5-6681e172fe1e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:09:23.765316 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.765289    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:23.765410 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.765323    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:23.765410 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.765337    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:23.765410 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.765350    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:23.765410 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.765364    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:23.766552 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.766529    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-envoy" pod "042f8c8d-a857-4efb-b8b5-6681e172fe1e" (UID: "042f8c8d-a857-4efb-b8b5-6681e172fe1e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:09:23.766865 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.766832    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-token" pod "042f8c8d-a857-4efb-b8b5-6681e172fe1e" (UID: "042f8c8d-a857-4efb-b8b5-6681e172fe1e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:09:23.766944 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.766871    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/042f8c8d-a857-4efb-b8b5-6681e172fe1e-kube-api-access-lpvkz" pod "042f8c8d-a857-4efb-b8b5-6681e172fe1e" (UID: "042f8c8d-a857-4efb-b8b5-6681e172fe1e"). InnerVolumeSpecName "kube-api-access-lpvkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:09:23.766944 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.766866    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-podinfo" pod "042f8c8d-a857-4efb-b8b5-6681e172fe1e" (UID: "042f8c8d-a857-4efb-b8b5-6681e172fe1e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:09:23.866469 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.866416    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:23.866469 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.866447    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:23.866469 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.866461    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/042f8c8d-a857-4efb-b8b5-6681e172fe1e-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:23.866469 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:23.866475    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpvkz\" (UniqueName: \"kubernetes.io/projected/042f8c8d-a857-4efb-b8b5-6681e172fe1e-kube-api-access-lpvkz\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:24.351929 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:24.351887    4945 generic.go:356] "Generic (PLEG): container finished" podID="042f8c8d-a857-4efb-b8b5-6681e172fe1e" containerID="381e1d22e2884c7b5830f65d5cc557c1d2b674d9945cb3739e05410387268828" exitCode=0
Oct 30 13:09:24.352107 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:24.351937    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"
Oct 30 13:09:24.352107 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:24.351975    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r" event={"ID":"042f8c8d-a857-4efb-b8b5-6681e172fe1e","Type":"ContainerDied","Data":"381e1d22e2884c7b5830f65d5cc557c1d2b674d9945cb3739e05410387268828"}
Oct 30 13:09:24.352107 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:24.352038    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r" event={"ID":"042f8c8d-a857-4efb-b8b5-6681e172fe1e","Type":"ContainerDied","Data":"176fc3933bc6e52810938b832158fa9d2c1d5ba079e0d7f616b096d9628115e9"}
Oct 30 13:09:24.352107 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:24.352064    4945 scope.go:122] "RemoveContainer" containerID="381e1d22e2884c7b5830f65d5cc557c1d2b674d9945cb3739e05410387268828"
Oct 30 13:09:24.362859 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:24.362838    4945 scope.go:122] "RemoveContainer" containerID="381e1d22e2884c7b5830f65d5cc557c1d2b674d9945cb3739e05410387268828"
Oct 30 13:09:24.363138 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:09:24.363119    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"381e1d22e2884c7b5830f65d5cc557c1d2b674d9945cb3739e05410387268828\": container with ID starting with 381e1d22e2884c7b5830f65d5cc557c1d2b674d9945cb3739e05410387268828 not found: ID does not exist" containerID="381e1d22e2884c7b5830f65d5cc557c1d2b674d9945cb3739e05410387268828"
Oct 30 13:09:24.363181 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:24.363166    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"381e1d22e2884c7b5830f65d5cc557c1d2b674d9945cb3739e05410387268828"} err="failed to get container status \"381e1d22e2884c7b5830f65d5cc557c1d2b674d9945cb3739e05410387268828\": rpc error: code = NotFound desc = could not find container \"381e1d22e2884c7b5830f65d5cc557c1d2b674d9945cb3739e05410387268828\": container with ID starting with 381e1d22e2884c7b5830f65d5cc557c1d2b674d9945cb3739e05410387268828 not found: ID does not exist"
Oct 30 13:09:24.417768 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:24.417732    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"]
Oct 30 13:09:24.428949 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:24.428917    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-4cb0fe-openshift-default-86c9c7d99c-fgr4r"]
Oct 30 13:09:25.560623 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:25.559637    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="042f8c8d-a857-4efb-b8b5-6681e172fe1e" path="/var/lib/kubelet/pods/042f8c8d-a857-4efb-b8b5-6681e172fe1e/volumes"
Oct 30 13:09:26.289677 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:09:26.283742    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:09:27.533727 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:27.533678    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:09:29.829120 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:29.829082    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"]
Oct 30 13:09:29.829527 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:29.829312    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr" podUID="bae96c60-5094-420b-8dd6-8bba39c588a8" containerName="maas-api" containerID="cri-o://9f0da851d16460dc5b2355d38975167d04ce2ae6fc6a9cfc51dfa90652ff392a" gracePeriod=30
Oct 30 13:09:30.145132 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.145101    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:09:30.233668 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.233631    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-metrics-certs\") pod \"bae96c60-5094-420b-8dd6-8bba39c588a8\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") "
Oct 30 13:09:30.233857 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.233711    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bae96c60-5094-420b-8dd6-8bba39c588a8-kube-api-access-czv4h\" (UniqueName: \"kubernetes.io/projected/bae96c60-5094-420b-8dd6-8bba39c588a8-kube-api-access-czv4h\") pod \"bae96c60-5094-420b-8dd6-8bba39c588a8\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") "
Oct 30 13:09:30.235819 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.235787    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-metrics-certs" pod "bae96c60-5094-420b-8dd6-8bba39c588a8" (UID: "bae96c60-5094-420b-8dd6-8bba39c588a8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:09:30.235943 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.235838    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bae96c60-5094-420b-8dd6-8bba39c588a8-kube-api-access-czv4h" pod "bae96c60-5094-420b-8dd6-8bba39c588a8" (UID: "bae96c60-5094-420b-8dd6-8bba39c588a8"). InnerVolumeSpecName "kube-api-access-czv4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:09:30.336383 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.336343    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-maas-api-tls\") pod \"bae96c60-5094-420b-8dd6-8bba39c588a8\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") "
Oct 30 13:09:30.336615 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.336407    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bae96c60-5094-420b-8dd6-8bba39c588a8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bae96c60-5094-420b-8dd6-8bba39c588a8-openshift-service-ca\") pod \"bae96c60-5094-420b-8dd6-8bba39c588a8\" (UID: \"bae96c60-5094-420b-8dd6-8bba39c588a8\") "
Oct 30 13:09:30.336800 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.336765    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:30.336800 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.336788    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czv4h\" (UniqueName: \"kubernetes.io/projected/bae96c60-5094-420b-8dd6-8bba39c588a8-kube-api-access-czv4h\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:30.337170 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.337146    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bae96c60-5094-420b-8dd6-8bba39c588a8-openshift-service-ca" pod "bae96c60-5094-420b-8dd6-8bba39c588a8" (UID: "bae96c60-5094-420b-8dd6-8bba39c588a8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:09:30.341585 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.341373    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-maas-api-tls" pod "bae96c60-5094-420b-8dd6-8bba39c588a8" (UID: "bae96c60-5094-420b-8dd6-8bba39c588a8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:09:30.437664 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.437555    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bae96c60-5094-420b-8dd6-8bba39c588a8-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:30.437664 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.437592    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bae96c60-5094-420b-8dd6-8bba39c588a8-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:30.579687 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.579648    4945 generic.go:356] "Generic (PLEG): container finished" podID="bae96c60-5094-420b-8dd6-8bba39c588a8" containerID="9f0da851d16460dc5b2355d38975167d04ce2ae6fc6a9cfc51dfa90652ff392a" exitCode=0
Oct 30 13:09:30.579687 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.579695    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"
Oct 30 13:09:30.579958 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.579732    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr" event={"ID":"bae96c60-5094-420b-8dd6-8bba39c588a8","Type":"ContainerDied","Data":"9f0da851d16460dc5b2355d38975167d04ce2ae6fc6a9cfc51dfa90652ff392a"}
Oct 30 13:09:30.579958 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.579795    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr" event={"ID":"bae96c60-5094-420b-8dd6-8bba39c588a8","Type":"ContainerDied","Data":"64870ce8ecc261a40eb3ef55af3e63d8ed57b6c9e5982ecd5fc31e29a3191272"}
Oct 30 13:09:30.579958 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.579824    4945 scope.go:122] "RemoveContainer" containerID="9f0da851d16460dc5b2355d38975167d04ce2ae6fc6a9cfc51dfa90652ff392a"
Oct 30 13:09:30.594236 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.594213    4945 scope.go:122] "RemoveContainer" containerID="9f0da851d16460dc5b2355d38975167d04ce2ae6fc6a9cfc51dfa90652ff392a"
Oct 30 13:09:30.594667 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:09:30.594635    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f0da851d16460dc5b2355d38975167d04ce2ae6fc6a9cfc51dfa90652ff392a\": container with ID starting with 9f0da851d16460dc5b2355d38975167d04ce2ae6fc6a9cfc51dfa90652ff392a not found: ID does not exist" containerID="9f0da851d16460dc5b2355d38975167d04ce2ae6fc6a9cfc51dfa90652ff392a"
Oct 30 13:09:30.594786 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.594679    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f0da851d16460dc5b2355d38975167d04ce2ae6fc6a9cfc51dfa90652ff392a"} err="failed to get container status \"9f0da851d16460dc5b2355d38975167d04ce2ae6fc6a9cfc51dfa90652ff392a\": rpc error: code = NotFound desc = could not find container \"9f0da851d16460dc5b2355d38975167d04ce2ae6fc6a9cfc51dfa90652ff392a\": container with ID starting with 9f0da851d16460dc5b2355d38975167d04ce2ae6fc6a9cfc51dfa90652ff392a not found: ID does not exist"
Oct 30 13:09:30.631311 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.631280    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"]
Oct 30 13:09:30.642391 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:30.642358    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-8f3cb8-7db76b9554-jx2sr"]
Oct 30 13:09:31.549608 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:31.549560    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bae96c60-5094-420b-8dd6-8bba39c588a8" path="/var/lib/kubelet/pods/bae96c60-5094-420b-8dd6-8bba39c588a8/volumes"
Oct 30 13:09:38.286012 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.285976    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"]
Oct 30 13:09:38.286849 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.286771    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw" podUID="a6ade70c-d9ac-4854-baae-0190ebd5ae08" containerName="maas-api" containerID="cri-o://cc253c42d801b0d0b669221426b4ad43b04dd7998abcca9e9a212d452b871a23" gracePeriod=30
Oct 30 13:09:38.630592 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.630560    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:09:38.632417 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.632390    4945 generic.go:356] "Generic (PLEG): container finished" podID="a6ade70c-d9ac-4854-baae-0190ebd5ae08" containerID="cc253c42d801b0d0b669221426b4ad43b04dd7998abcca9e9a212d452b871a23" exitCode=0
Oct 30 13:09:38.632574 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.632420    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw" event={"ID":"a6ade70c-d9ac-4854-baae-0190ebd5ae08","Type":"ContainerDied","Data":"cc253c42d801b0d0b669221426b4ad43b04dd7998abcca9e9a212d452b871a23"}
Oct 30 13:09:38.632574 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.632442    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"
Oct 30 13:09:38.632574 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.632476    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw" event={"ID":"a6ade70c-d9ac-4854-baae-0190ebd5ae08","Type":"ContainerDied","Data":"c72907e67d0abf954e0fe03f7342bebe07867289c4137d6d932d2254a36152f2"}
Oct 30 13:09:38.632574 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.632501    4945 scope.go:122] "RemoveContainer" containerID="cc253c42d801b0d0b669221426b4ad43b04dd7998abcca9e9a212d452b871a23"
Oct 30 13:09:38.651996 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.651974    4945 scope.go:122] "RemoveContainer" containerID="cc253c42d801b0d0b669221426b4ad43b04dd7998abcca9e9a212d452b871a23"
Oct 30 13:09:38.652302 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:09:38.652274    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc253c42d801b0d0b669221426b4ad43b04dd7998abcca9e9a212d452b871a23\": container with ID starting with cc253c42d801b0d0b669221426b4ad43b04dd7998abcca9e9a212d452b871a23 not found: ID does not exist" containerID="cc253c42d801b0d0b669221426b4ad43b04dd7998abcca9e9a212d452b871a23"
Oct 30 13:09:38.652381 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.652310    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc253c42d801b0d0b669221426b4ad43b04dd7998abcca9e9a212d452b871a23"} err="failed to get container status \"cc253c42d801b0d0b669221426b4ad43b04dd7998abcca9e9a212d452b871a23\": rpc error: code = NotFound desc = could not find container \"cc253c42d801b0d0b669221426b4ad43b04dd7998abcca9e9a212d452b871a23\": container with ID starting with cc253c42d801b0d0b669221426b4ad43b04dd7998abcca9e9a212d452b871a23 not found: ID does not exist"
Oct 30 13:09:38.732638 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.732581    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a6ade70c-d9ac-4854-baae-0190ebd5ae08-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6ade70c-d9ac-4854-baae-0190ebd5ae08-metrics-certs\") pod \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") "
Oct 30 13:09:38.732817 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.732670    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6ade70c-d9ac-4854-baae-0190ebd5ae08-kube-api-access-9mk9z\" (UniqueName: \"kubernetes.io/projected/a6ade70c-d9ac-4854-baae-0190ebd5ae08-kube-api-access-9mk9z\") pod \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") "
Oct 30 13:09:38.732876 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.732850    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a6ade70c-d9ac-4854-baae-0190ebd5ae08-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6ade70c-d9ac-4854-baae-0190ebd5ae08-maas-api-tls\") pod \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") "
Oct 30 13:09:38.732954 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.732924    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a6ade70c-d9ac-4854-baae-0190ebd5ae08-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6ade70c-d9ac-4854-baae-0190ebd5ae08-openshift-service-ca\") pod \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\" (UID: \"a6ade70c-d9ac-4854-baae-0190ebd5ae08\") "
Oct 30 13:09:38.733357 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.733326    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6ade70c-d9ac-4854-baae-0190ebd5ae08-openshift-service-ca" pod "a6ade70c-d9ac-4854-baae-0190ebd5ae08" (UID: "a6ade70c-d9ac-4854-baae-0190ebd5ae08"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:09:38.734836 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.734803    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6ade70c-d9ac-4854-baae-0190ebd5ae08-metrics-certs" pod "a6ade70c-d9ac-4854-baae-0190ebd5ae08" (UID: "a6ade70c-d9ac-4854-baae-0190ebd5ae08"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:09:38.734935 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.734808    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6ade70c-d9ac-4854-baae-0190ebd5ae08-kube-api-access-9mk9z" pod "a6ade70c-d9ac-4854-baae-0190ebd5ae08" (UID: "a6ade70c-d9ac-4854-baae-0190ebd5ae08"). InnerVolumeSpecName "kube-api-access-9mk9z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:09:38.734935 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.734887    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6ade70c-d9ac-4854-baae-0190ebd5ae08-maas-api-tls" pod "a6ade70c-d9ac-4854-baae-0190ebd5ae08" (UID: "a6ade70c-d9ac-4854-baae-0190ebd5ae08"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:09:38.843502 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.843448    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6ade70c-d9ac-4854-baae-0190ebd5ae08-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:38.843502 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.843492    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a6ade70c-d9ac-4854-baae-0190ebd5ae08-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:38.843502 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.843507    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mk9z\" (UniqueName: \"kubernetes.io/projected/a6ade70c-d9ac-4854-baae-0190ebd5ae08-kube-api-access-9mk9z\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:38.843877 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.843521    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6ade70c-d9ac-4854-baae-0190ebd5ae08-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:09:38.982259 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.982224    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"]
Oct 30 13:09:38.987400 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:38.987372    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-a81a48-d5595c8cf-vnltw"]
Oct 30 13:09:39.549161 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:09:39.549126    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a6ade70c-d9ac-4854-baae-0190ebd5ae08" path="/var/lib/kubelet/pods/a6ade70c-d9ac-4854-baae-0190ebd5ae08/volumes"
Oct 30 13:09:56.195076 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:09:56.195035    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:10:00.579622 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:00.578702    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:10:05.919291 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:05.919254    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"]
Oct 30 13:10:05.930209 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:05.930172    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"]
Oct 30 13:10:05.936104 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:05.936063    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"]
Oct 30 13:10:20.944111 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:20.944047    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" containerID="cri-o://aa901b2bbc3a613c7e582cf86b81a687999655f324ad6a432c67737fefafad5b" gracePeriod=45
Oct 30 13:10:20.958999 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:20.958962    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main" containerID="cri-o://5e49c961322088e3f44fabad7357e18fe9c29383abccd865804c0ed0c0402713" gracePeriod=45
Oct 30 13:10:20.959121 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:20.959048    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main" containerID="cri-o://0e7735c46ecb3cc5627a5d794a086a8b6a3b712d99b0b6b05553030bea8367d4" gracePeriod=45
Oct 30 13:10:21.189007 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.188915    4945 generic.go:356] "Generic (PLEG): container finished" podID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerID="0e7735c46ecb3cc5627a5d794a086a8b6a3b712d99b0b6b05553030bea8367d4" exitCode=0
Oct 30 13:10:21.189007 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.188985    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" event={"ID":"0a699888-47fc-44b7-88fb-26b4a0efecba","Type":"ContainerDied","Data":"0e7735c46ecb3cc5627a5d794a086a8b6a3b712d99b0b6b05553030bea8367d4"}
Oct 30 13:10:21.191147 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.191120    4945 generic.go:356] "Generic (PLEG): container finished" podID="f3bc8355-ae84-4811-935f-0046ec42375e" containerID="aa901b2bbc3a613c7e582cf86b81a687999655f324ad6a432c67737fefafad5b" exitCode=0
Oct 30 13:10:21.191375 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.191329    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" event={"ID":"f3bc8355-ae84-4811-935f-0046ec42375e","Type":"ContainerDied","Data":"aa901b2bbc3a613c7e582cf86b81a687999655f324ad6a432c67737fefafad5b"}
Oct 30 13:10:21.193313 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.193218    4945 generic.go:356] "Generic (PLEG): container finished" podID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerID="5e49c961322088e3f44fabad7357e18fe9c29383abccd865804c0ed0c0402713" exitCode=0
Oct 30 13:10:21.193313 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.193284    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" event={"ID":"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa","Type":"ContainerDied","Data":"5e49c961322088e3f44fabad7357e18fe9c29383abccd865804c0ed0c0402713"}
Oct 30 13:10:21.293867 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.293830    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 13:10:21.313615 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.305329    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 13:10:21.313615 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.312422    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.415924    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-tmp-dir\") pod \"f3bc8355-ae84-4811-935f-0046ec42375e\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.415994    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-home\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-home\") pod \"0a699888-47fc-44b7-88fb-26b4a0efecba\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416038    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-dshm\") pod \"f3bc8355-ae84-4811-935f-0046ec42375e\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416079    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-tmp-dir\") pod \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416157    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-model-cache\") pod \"f3bc8355-ae84-4811-935f-0046ec42375e\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416194    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-kube-api-access-flp5b\" (UniqueName: \"kubernetes.io/projected/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-kube-api-access-flp5b\") pod \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416250    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-model-cache\") pod \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416290    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3bc8355-ae84-4811-935f-0046ec42375e-kube-api-access-dvmzt\" (UniqueName: \"kubernetes.io/projected/f3bc8355-ae84-4811-935f-0046ec42375e-kube-api-access-dvmzt\") pod \"f3bc8355-ae84-4811-935f-0046ec42375e\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416351    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-home\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-home\") pod \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416401    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-home\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-home\") pod \"f3bc8355-ae84-4811-935f-0046ec42375e\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416444    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a699888-47fc-44b7-88fb-26b4a0efecba-kube-api-access-plmpv\" (UniqueName: \"kubernetes.io/projected/0a699888-47fc-44b7-88fb-26b4a0efecba-kube-api-access-plmpv\") pod \"0a699888-47fc-44b7-88fb-26b4a0efecba\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416477    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-dshm\") pod \"0a699888-47fc-44b7-88fb-26b4a0efecba\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416538    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-dshm\") pod \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416622    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0a699888-47fc-44b7-88fb-26b4a0efecba-tls-certs\" (UniqueName: \"kubernetes.io/secret/0a699888-47fc-44b7-88fb-26b4a0efecba-tls-certs\") pod \"0a699888-47fc-44b7-88fb-26b4a0efecba\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416666    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-tmp-dir\") pod \"0a699888-47fc-44b7-88fb-26b4a0efecba\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416697    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f3bc8355-ae84-4811-935f-0046ec42375e-tls-certs\" (UniqueName: \"kubernetes.io/secret/f3bc8355-ae84-4811-935f-0046ec42375e-tls-certs\") pod \"f3bc8355-ae84-4811-935f-0046ec42375e\" (UID: \"f3bc8355-ae84-4811-935f-0046ec42375e\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416736    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-model-cache\") pod \"0a699888-47fc-44b7-88fb-26b4a0efecba\" (UID: \"0a699888-47fc-44b7-88fb-26b4a0efecba\") "
Oct 30 13:10:21.417572 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.416769    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-tls-certs\" (UniqueName: \"kubernetes.io/secret/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-tls-certs\") pod \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\" (UID: \"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa\") "
Oct 30 13:10:21.420508 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.420230    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-tmp-dir" pod "0a699888-47fc-44b7-88fb-26b4a0efecba" (UID: "0a699888-47fc-44b7-88fb-26b4a0efecba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:21.420983 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.420717    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-tls-certs" pod "1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" (UID: "1a1454c3-5b47-4a17-8dd9-dc9ba7735caa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:10:21.421212 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.421133    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-model-cache" pod "0a699888-47fc-44b7-88fb-26b4a0efecba" (UID: "0a699888-47fc-44b7-88fb-26b4a0efecba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:21.421738 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.421546    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-home" pod "f3bc8355-ae84-4811-935f-0046ec42375e" (UID: "f3bc8355-ae84-4811-935f-0046ec42375e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:21.422057 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.421880    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-tmp-dir" pod "1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" (UID: "1a1454c3-5b47-4a17-8dd9-dc9ba7735caa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:21.422368 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.422197    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-tmp-dir" pod "f3bc8355-ae84-4811-935f-0046ec42375e" (UID: "f3bc8355-ae84-4811-935f-0046ec42375e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:21.422520 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.422499    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-home" pod "0a699888-47fc-44b7-88fb-26b4a0efecba" (UID: "0a699888-47fc-44b7-88fb-26b4a0efecba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:21.423174 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.423153    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-home" pod "1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" (UID: "1a1454c3-5b47-4a17-8dd9-dc9ba7735caa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:21.423487 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.423467    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-model-cache" pod "f3bc8355-ae84-4811-935f-0046ec42375e" (UID: "f3bc8355-ae84-4811-935f-0046ec42375e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:21.424854 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.424831    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-dshm" pod "f3bc8355-ae84-4811-935f-0046ec42375e" (UID: "f3bc8355-ae84-4811-935f-0046ec42375e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:21.425219 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.425196    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-model-cache" pod "1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" (UID: "1a1454c3-5b47-4a17-8dd9-dc9ba7735caa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:21.426489 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.426462    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3bc8355-ae84-4811-935f-0046ec42375e-tls-certs" pod "f3bc8355-ae84-4811-935f-0046ec42375e" (UID: "f3bc8355-ae84-4811-935f-0046ec42375e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:10:21.426797 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.426774    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-dshm" pod "1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" (UID: "1a1454c3-5b47-4a17-8dd9-dc9ba7735caa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:21.427001 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.426977    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-kube-api-access-flp5b" pod "1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" (UID: "1a1454c3-5b47-4a17-8dd9-dc9ba7735caa"). InnerVolumeSpecName "kube-api-access-flp5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:21.428842 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.428792    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a699888-47fc-44b7-88fb-26b4a0efecba-tls-certs" pod "0a699888-47fc-44b7-88fb-26b4a0efecba" (UID: "0a699888-47fc-44b7-88fb-26b4a0efecba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:10:21.429178 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.429054    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3bc8355-ae84-4811-935f-0046ec42375e-kube-api-access-dvmzt" pod "f3bc8355-ae84-4811-935f-0046ec42375e" (UID: "f3bc8355-ae84-4811-935f-0046ec42375e"). InnerVolumeSpecName "kube-api-access-dvmzt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:21.429178 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.429151    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a699888-47fc-44b7-88fb-26b4a0efecba-kube-api-access-plmpv" pod "0a699888-47fc-44b7-88fb-26b4a0efecba" (UID: "0a699888-47fc-44b7-88fb-26b4a0efecba"). InnerVolumeSpecName "kube-api-access-plmpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:21.429707 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.429673    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-dshm" pod "0a699888-47fc-44b7-88fb-26b4a0efecba" (UID: "0a699888-47fc-44b7-88fb-26b4a0efecba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:21.518421 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518336    4945 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-home\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518421 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518367    4945 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-dshm\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518421 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518376    4945 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-tmp-dir\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518421 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518385    4945 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-model-cache\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518421 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518395    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flp5b\" (UniqueName: \"kubernetes.io/projected/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-kube-api-access-flp5b\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518421 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518404    4945 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-model-cache\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518421 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518412    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvmzt\" (UniqueName: \"kubernetes.io/projected/f3bc8355-ae84-4811-935f-0046ec42375e-kube-api-access-dvmzt\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518421 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518421    4945 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-home\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518421 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518428    4945 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-home\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518854 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518436    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plmpv\" (UniqueName: \"kubernetes.io/projected/0a699888-47fc-44b7-88fb-26b4a0efecba-kube-api-access-plmpv\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518854 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518443    4945 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-dshm\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518854 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518451    4945 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-dshm\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518854 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518458    4945 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a699888-47fc-44b7-88fb-26b4a0efecba-tls-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518854 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518465    4945 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-tmp-dir\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518854 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518472    4945 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3bc8355-ae84-4811-935f-0046ec42375e-tls-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518854 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518480    4945 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a699888-47fc-44b7-88fb-26b4a0efecba-model-cache\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518854 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518487    4945 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa-tls-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:21.518854 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:21.518493    4945 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3bc8355-ae84-4811-935f-0046ec42375e-tmp-dir\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:22.201862 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.201833    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"
Oct 30 13:10:22.203202 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.201885    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh" event={"ID":"1a1454c3-5b47-4a17-8dd9-dc9ba7735caa","Type":"ContainerDied","Data":"b58077487caf479458785f6789370b566b7357a91c21f8eb2997ee10d9864349"}
Oct 30 13:10:22.203345 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.203235    4945 scope.go:122] "RemoveContainer" containerID="5e49c961322088e3f44fabad7357e18fe9c29383abccd865804c0ed0c0402713"
Oct 30 13:10:22.204083 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.204026    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"
Oct 30 13:10:22.204083 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.204036    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h" event={"ID":"0a699888-47fc-44b7-88fb-26b4a0efecba","Type":"ContainerDied","Data":"548450e24277a501171ad82b866e4e0b28309fbbd4c5a2b2965e490ca93580fe"}
Oct 30 13:10:22.209167 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.208412    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" event={"ID":"f3bc8355-ae84-4811-935f-0046ec42375e","Type":"ContainerDied","Data":"a9507cd7e236d1b9d48072b536fc9f5f1e0bc582b471318f98a4e9828b2b1c4e"}
Oct 30 13:10:22.209167 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.208496    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"
Oct 30 13:10:22.219409 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.219386    4945 scope.go:122] "RemoveContainer" containerID="0e7735c46ecb3cc5627a5d794a086a8b6a3b712d99b0b6b05553030bea8367d4"
Oct 30 13:10:22.233464 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.233421    4945 scope.go:122] "RemoveContainer" containerID="aa901b2bbc3a613c7e582cf86b81a687999655f324ad6a432c67737fefafad5b"
Oct 30 13:10:22.256832 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.256798    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"]
Oct 30 13:10:22.266586 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.266560    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6"]
Oct 30 13:10:22.276666 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.276633    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-7a590a/distinct-w1-7a590a-kserve-9d68ddd45-fb9d6" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 13:10:22.294429 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.294396    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"]
Oct 30 13:10:22.300486 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.300458    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/premium-w1-7a590a-kserve-5cb8dddf94-8dhkh"]
Oct 30 13:10:22.337152 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.337115    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"]
Oct 30 13:10:22.341283 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:22.341250    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-7a590a/simulator-w1-7a590a-kserve-676464b495-kq85h"]
Oct 30 13:10:23.554960 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:23.554920    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" path="/var/lib/kubelet/pods/0a699888-47fc-44b7-88fb-26b4a0efecba/volumes"
Oct 30 13:10:23.556214 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:23.556188    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" path="/var/lib/kubelet/pods/1a1454c3-5b47-4a17-8dd9-dc9ba7735caa/volumes"
Oct 30 13:10:23.556976 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:23.556955    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" path="/var/lib/kubelet/pods/f3bc8355-ae84-4811-935f-0046ec42375e/volumes"
Oct 30 13:10:26.195100 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:10:26.195041    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:10:28.956954 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:28.956912    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"]
Oct 30 13:10:28.957651 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:28.957181    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p" podUID="15f559f9-e4fa-43f3-8308-a0c9a947080c" containerName="maas-api" containerID="cri-o://d5a97327a36cf488c6a84349c9fee4cdce5040ac0202e8c5d8f6b2bb253cdbbc" gracePeriod=30
Oct 30 13:10:30.222681 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.222647    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 13:10:30.333191 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.333108    4945 generic.go:356] "Generic (PLEG): container finished" podID="15f559f9-e4fa-43f3-8308-a0c9a947080c" containerID="d5a97327a36cf488c6a84349c9fee4cdce5040ac0202e8c5d8f6b2bb253cdbbc" exitCode=0
Oct 30 13:10:30.333191 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.333165    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"
Oct 30 13:10:30.333430 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.333197    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p" event={"ID":"15f559f9-e4fa-43f3-8308-a0c9a947080c","Type":"ContainerDied","Data":"d5a97327a36cf488c6a84349c9fee4cdce5040ac0202e8c5d8f6b2bb253cdbbc"}
Oct 30 13:10:30.333430 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.333272    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p" event={"ID":"15f559f9-e4fa-43f3-8308-a0c9a947080c","Type":"ContainerDied","Data":"2fe8d9c8d7f9bc47614dc63eb31425e8a3aeca0bb11e9f9fe6015eb45f3f7b45"}
Oct 30 13:10:30.333430 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.333317    4945 scope.go:122] "RemoveContainer" containerID="d5a97327a36cf488c6a84349c9fee4cdce5040ac0202e8c5d8f6b2bb253cdbbc"
Oct 30 13:10:30.342958 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.342939    4945 scope.go:122] "RemoveContainer" containerID="d5a97327a36cf488c6a84349c9fee4cdce5040ac0202e8c5d8f6b2bb253cdbbc"
Oct 30 13:10:30.343239 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:10:30.343220    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5a97327a36cf488c6a84349c9fee4cdce5040ac0202e8c5d8f6b2bb253cdbbc\": container with ID starting with d5a97327a36cf488c6a84349c9fee4cdce5040ac0202e8c5d8f6b2bb253cdbbc not found: ID does not exist" containerID="d5a97327a36cf488c6a84349c9fee4cdce5040ac0202e8c5d8f6b2bb253cdbbc"
Oct 30 13:10:30.343294 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.343250    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a97327a36cf488c6a84349c9fee4cdce5040ac0202e8c5d8f6b2bb253cdbbc"} err="failed to get container status \"d5a97327a36cf488c6a84349c9fee4cdce5040ac0202e8c5d8f6b2bb253cdbbc\": rpc error: code = NotFound desc = could not find container \"d5a97327a36cf488c6a84349c9fee4cdce5040ac0202e8c5d8f6b2bb253cdbbc\": container with ID starting with d5a97327a36cf488c6a84349c9fee4cdce5040ac0202e8c5d8f6b2bb253cdbbc not found: ID does not exist"
Oct 30 13:10:30.363778 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.363754    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/15f559f9-e4fa-43f3-8308-a0c9a947080c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15f559f9-e4fa-43f3-8308-a0c9a947080c-openshift-service-ca\") pod \"15f559f9-e4fa-43f3-8308-a0c9a947080c\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") "
Oct 30 13:10:30.363910 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.363812    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-metrics-certs\") pod \"15f559f9-e4fa-43f3-8308-a0c9a947080c\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") "
Oct 30 13:10:30.365438 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.364348    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-maas-api-tls\") pod \"15f559f9-e4fa-43f3-8308-a0c9a947080c\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") "
Oct 30 13:10:30.365438 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.364386    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15f559f9-e4fa-43f3-8308-a0c9a947080c-openshift-service-ca" pod "15f559f9-e4fa-43f3-8308-a0c9a947080c" (UID: "15f559f9-e4fa-43f3-8308-a0c9a947080c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:10:30.365438 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.364411    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/15f559f9-e4fa-43f3-8308-a0c9a947080c-kube-api-access-mpfpk\" (UniqueName: \"kubernetes.io/projected/15f559f9-e4fa-43f3-8308-a0c9a947080c-kube-api-access-mpfpk\") pod \"15f559f9-e4fa-43f3-8308-a0c9a947080c\" (UID: \"15f559f9-e4fa-43f3-8308-a0c9a947080c\") "
Oct 30 13:10:30.365787 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.365534    4945 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15f559f9-e4fa-43f3-8308-a0c9a947080c-openshift-service-ca\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:30.402955 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.402894    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-maas-api-tls" pod "15f559f9-e4fa-43f3-8308-a0c9a947080c" (UID: "15f559f9-e4fa-43f3-8308-a0c9a947080c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:10:30.406806 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.406732    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-metrics-certs" pod "15f559f9-e4fa-43f3-8308-a0c9a947080c" (UID: "15f559f9-e4fa-43f3-8308-a0c9a947080c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 13:10:30.409495 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.409427    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15f559f9-e4fa-43f3-8308-a0c9a947080c-kube-api-access-mpfpk" pod "15f559f9-e4fa-43f3-8308-a0c9a947080c" (UID: "15f559f9-e4fa-43f3-8308-a0c9a947080c"). InnerVolumeSpecName "kube-api-access-mpfpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:30.467132 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.467094    4945 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-metrics-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:30.467132 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.467127    4945 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15f559f9-e4fa-43f3-8308-a0c9a947080c-maas-api-tls\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:30.467132 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.467138    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpfpk\" (UniqueName: \"kubernetes.io/projected/15f559f9-e4fa-43f3-8308-a0c9a947080c-kube-api-access-mpfpk\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:30.679932 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.679892    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"]
Oct 30 13:10:30.691708 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:30.691673    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-53dcb1-54896cf466-4gq8p"]
Oct 30 13:10:31.549884 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:31.549849    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="15f559f9-e4fa-43f3-8308-a0c9a947080c" path="/var/lib/kubelet/pods/15f559f9-e4fa-43f3-8308-a0c9a947080c/volumes"
Oct 30 13:10:37.241039 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:37.241004    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"]
Oct 30 13:10:37.241860 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:37.241804    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerName="istio-proxy" containerID="cri-o://f64f4e18b1dae7ea1caa7e05a2cb5bdc0379308b1318b00d0ca1c6d4abbc64b7" gracePeriod=30
Oct 30 13:10:37.253693 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:37.253651    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"]
Oct 30 13:10:37.254047 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:37.254001    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" podUID="473f256f-7ee3-4e49-b407-346e0242fcbb" containerName="istio-proxy" containerID="cri-o://8fbbe50b7d48c9089db28460562410ad41175fd9c24f0c5e69166d2dbd686a73" gracePeriod=30
Oct 30 13:10:37.257283 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:37.257258    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"]
Oct 30 13:10:37.257787 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:37.257722    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw" podUID="b87eaba7-33de-46f3-a25d-2e86f1e12abe" containerName="istio-proxy" containerID="cri-o://942bad7daf144f602f87b78beb1b99bfeda4ebb86ae9d17a901c1dfa6ab2b6e5" gracePeriod=30
Oct 30 13:10:37.496672 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:37.496286    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"]
Oct 30 13:10:37.496672 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:37.496619    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9" podUID="237869eb-18ff-4f5a-87e8-e7d8c97bd088" containerName="istio-proxy" containerID="cri-o://8185dd7b9cc870ecf6a1ad499fec7157df1842ea6f9aa7f81f2b79ca5388f322" gracePeriod=30
Oct 30 13:10:37.504641 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:37.504611    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"]
Oct 30 13:10:37.505087 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:37.505034    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk" podUID="492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" containerName="istio-proxy" containerID="cri-o://c0341fe52dcfb8e1d994335f8acfd2da7b613f0e3ea4711a740fb11d93216843" gracePeriod=30
Oct 30 13:10:37.525576 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:37.525506    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"]
Oct 30 13:10:37.525901 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:37.525871    4945 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd" podUID="b107d489-5cd0-492b-8cab-01d603040f8b" containerName="istio-proxy" containerID="cri-o://63b413b7ea2f4104826307b87408e1df9b9589795c2bec913e508474c4db6863" gracePeriod=30
Oct 30 13:10:38.793505 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:38.793473    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-779bff956b-l69xd_f6d044ab-287d-434c-8fd5-bb9b61eaffaa/manager/0.log"
Oct 30 13:10:38.991501 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:38.991470    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_developer-portal-controller-66d8dbb8c8-jxp7q_2b240210-5d58-4791-8745-60c58f3e9bbf/manager/0.log"
Oct 30 13:10:39.189702 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:39.189587    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-57f5c66455-9g9vc_e0821ac0-2e9b-4403-914a-e13b7130e8a2/manager/0.log"
Oct 30 13:10:39.657189 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:39.657147    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:10:39.657377 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:39.657211    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:10:39.735152 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:39.735104    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:10:39.735304 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:39.735158    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw" podUID="b87eaba7-33de-46f3-a25d-2e86f1e12abe" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:10:39.989857 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:39.989816    4945 patch_prober.go:29] interesting pod/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 13:10:39.990443 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:39.989902    4945 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" podUID="473f256f-7ee3-4e49-b407-346e0242fcbb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 13:10:40.009804 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:40.009770    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6b575997cc-7g9t5_607ad48d-3968-4ffd-973f-51272eb00937/manager/0.log"
Oct 30 13:10:40.639558 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:40.639525    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_mcp-gateway-controller-6c98b995c5-j98z2_a59f98de-82e6-4a91-bdab-beb39922f5ff/mcp-controller/0.log"
Oct 30 13:10:41.602196 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:41.602161    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-5k947_55710abf-0a0a-4830-a4dd-54083ab2ac04/manager/0.log"
Oct 30 13:10:41.810304 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:41.810264    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-s9qks_85cd2a7e-fce3-43f2-8d41-feaf9406baa5/manager/0.log"
Oct 30 13:10:42.021461 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.021429    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-z228w_fbcc3687-0c22-414a-aacc-5f18a80f7d41/manager/0.log"
Oct 30 13:10:42.430402 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.430309    4945 generic.go:356] "Generic (PLEG): container finished" podID="473f256f-7ee3-4e49-b407-346e0242fcbb" containerID="8fbbe50b7d48c9089db28460562410ad41175fd9c24f0c5e69166d2dbd686a73" exitCode=0
Oct 30 13:10:42.430585 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.430460    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" event={"ID":"473f256f-7ee3-4e49-b407-346e0242fcbb","Type":"ContainerDied","Data":"8fbbe50b7d48c9089db28460562410ad41175fd9c24f0c5e69166d2dbd686a73"}
Oct 30 13:10:42.432492 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.432467    4945 generic.go:356] "Generic (PLEG): container finished" podID="b87eaba7-33de-46f3-a25d-2e86f1e12abe" containerID="942bad7daf144f602f87b78beb1b99bfeda4ebb86ae9d17a901c1dfa6ab2b6e5" exitCode=0
Oct 30 13:10:42.432674 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.432640    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw" event={"ID":"b87eaba7-33de-46f3-a25d-2e86f1e12abe","Type":"ContainerDied","Data":"942bad7daf144f602f87b78beb1b99bfeda4ebb86ae9d17a901c1dfa6ab2b6e5"}
Oct 30 13:10:42.434423 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.434398    4945 generic.go:356] "Generic (PLEG): container finished" podID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerID="f64f4e18b1dae7ea1caa7e05a2cb5bdc0379308b1318b00d0ca1c6d4abbc64b7" exitCode=0
Oct 30 13:10:42.434521 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.434451    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" event={"ID":"26fbb81d-799f-47c7-97ab-c5a0e30425ac","Type":"ContainerDied","Data":"f64f4e18b1dae7ea1caa7e05a2cb5bdc0379308b1318b00d0ca1c6d4abbc64b7"}
Oct 30 13:10:42.499921 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.499830    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/1.log"
Oct 30 13:10:42.671150 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.671064    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gzpff_b818f62c-1645-4782-93a6-da2341cdcce7/server/0.log"
Oct 30 13:10:42.710382 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.705982    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 13:10:42.712473 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.712389    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 13:10:42.716985 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.716967    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 13:10:42.724343 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.723963    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-token\" (UniqueName: \"kubernetes.io/projected/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-token\") pod \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") "
Oct 30 13:10:42.724343 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.724025    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-credential-socket\") pod \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") "
Oct 30 13:10:42.724343 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.724060    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-data\") pod \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") "
Oct 30 13:10:42.724343 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.724112    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-workload-socket\") pod \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") "
Oct 30 13:10:42.724343 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.724147    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-envoy\") pod \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") "
Oct 30 13:10:42.724343 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.724226    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26fbb81d-799f-47c7-97ab-c5a0e30425ac-kube-api-access-fdbht\" (UniqueName: \"kubernetes.io/projected/26fbb81d-799f-47c7-97ab-c5a0e30425ac-kube-api-access-fdbht\") pod \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") "
Oct 30 13:10:42.724343 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.724284    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-workload-certs\") pod \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") "
Oct 30 13:10:42.724343 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.724312    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istiod-ca-cert\") pod \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") "
Oct 30 13:10:42.724867 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.724388    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-podinfo\") pod \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\" (UID: \"26fbb81d-799f-47c7-97ab-c5a0e30425ac\") "
Oct 30 13:10:42.726691 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.726063    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-data" pod "26fbb81d-799f-47c7-97ab-c5a0e30425ac" (UID: "26fbb81d-799f-47c7-97ab-c5a0e30425ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.726691 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.726288    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-workload-certs" pod "26fbb81d-799f-47c7-97ab-c5a0e30425ac" (UID: "26fbb81d-799f-47c7-97ab-c5a0e30425ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.726691 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.726520    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-workload-socket" pod "26fbb81d-799f-47c7-97ab-c5a0e30425ac" (UID: "26fbb81d-799f-47c7-97ab-c5a0e30425ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.726691 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.726560    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-credential-socket" pod "26fbb81d-799f-47c7-97ab-c5a0e30425ac" (UID: "26fbb81d-799f-47c7-97ab-c5a0e30425ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.726988 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.726691    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istiod-ca-cert" pod "26fbb81d-799f-47c7-97ab-c5a0e30425ac" (UID: "26fbb81d-799f-47c7-97ab-c5a0e30425ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:10:42.729845 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.729819    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-token" pod "26fbb81d-799f-47c7-97ab-c5a0e30425ac" (UID: "26fbb81d-799f-47c7-97ab-c5a0e30425ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:42.731347 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.731252    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26fbb81d-799f-47c7-97ab-c5a0e30425ac-kube-api-access-fdbht" pod "26fbb81d-799f-47c7-97ab-c5a0e30425ac" (UID: "26fbb81d-799f-47c7-97ab-c5a0e30425ac"). InnerVolumeSpecName "kube-api-access-fdbht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:42.731347 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.731323    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-envoy" pod "26fbb81d-799f-47c7-97ab-c5a0e30425ac" (UID: "26fbb81d-799f-47c7-97ab-c5a0e30425ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.731723 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.731592    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-podinfo" pod "26fbb81d-799f-47c7-97ab-c5a0e30425ac" (UID: "26fbb81d-799f-47c7-97ab-c5a0e30425ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:10:42.790173 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.790117    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 13:10:42.825438 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.825405    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-credential-socket\") pod \"473f256f-7ee3-4e49-b407-346e0242fcbb\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") "
Oct 30 13:10:42.825613 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.825464    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/473f256f-7ee3-4e49-b407-346e0242fcbb-kube-api-access-dvm6v\" (UniqueName: \"kubernetes.io/projected/473f256f-7ee3-4e49-b407-346e0242fcbb-kube-api-access-dvm6v\") pod \"473f256f-7ee3-4e49-b407-346e0242fcbb\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") "
Oct 30 13:10:42.825613 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.825482    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-workload-socket\") pod \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") "
Oct 30 13:10:42.825613 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.825525    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-data\") pod \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") "
Oct 30 13:10:42.825613 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.825566    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-envoy\") pod \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") "
Oct 30 13:10:42.825851 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.825633    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-envoy\") pod \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") "
Oct 30 13:10:42.825851 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.825681    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-podinfo\") pod \"473f256f-7ee3-4e49-b407-346e0242fcbb\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") "
Oct 30 13:10:42.825851 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.825738    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-workload-socket\") pod \"473f256f-7ee3-4e49-b407-346e0242fcbb\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") "
Oct 30 13:10:42.825851 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.825770    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-credential-socket" pod "473f256f-7ee3-4e49-b407-346e0242fcbb" (UID: "473f256f-7ee3-4e49-b407-346e0242fcbb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.825851 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.825780    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/237869eb-18ff-4f5a-87e8-e7d8c97bd088-kube-api-access-kldn2\" (UniqueName: \"kubernetes.io/projected/237869eb-18ff-4f5a-87e8-e7d8c97bd088-kube-api-access-kldn2\") pod \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") "
Oct 30 13:10:42.826114 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.825875    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b87eaba7-33de-46f3-a25d-2e86f1e12abe-kube-api-access-r46dz\" (UniqueName: \"kubernetes.io/projected/b87eaba7-33de-46f3-a25d-2e86f1e12abe-kube-api-access-r46dz\") pod \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") "
Oct 30 13:10:42.826114 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.825918    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istiod-ca-cert\") pod \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") "
Oct 30 13:10:42.826114 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.825953    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-credential-socket\") pod \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") "
Oct 30 13:10:42.826114 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.825988    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istiod-ca-cert\") pod \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") "
Oct 30 13:10:42.826114 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.826022    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-workload-certs\") pod \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") "
Oct 30 13:10:42.826114 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.826075    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/473f256f-7ee3-4e49-b407-346e0242fcbb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/473f256f-7ee3-4e49-b407-346e0242fcbb-istiod-ca-cert\") pod \"473f256f-7ee3-4e49-b407-346e0242fcbb\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") "
Oct 30 13:10:42.826114 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.826106    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-data\") pod \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") "
Oct 30 13:10:42.826469 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.826144    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-token\" (UniqueName: \"kubernetes.io/projected/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-token\") pod \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") "
Oct 30 13:10:42.826469 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.826178    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-token\" (UniqueName: \"kubernetes.io/projected/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-token\") pod \"473f256f-7ee3-4e49-b407-346e0242fcbb\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") "
Oct 30 13:10:42.826469 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.826244    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-workload-certs\") pod \"473f256f-7ee3-4e49-b407-346e0242fcbb\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") "
Oct 30 13:10:42.826469 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.826274    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-workload-certs\") pod \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") "
Oct 30 13:10:42.826469 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.826332    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-envoy\") pod \"473f256f-7ee3-4e49-b407-346e0242fcbb\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") "
Oct 30 13:10:42.826469 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.826362    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-workload-socket\") pod \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") "
Oct 30 13:10:42.826469 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.826401    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-data\") pod \"473f256f-7ee3-4e49-b407-346e0242fcbb\" (UID: \"473f256f-7ee3-4e49-b407-346e0242fcbb\") "
Oct 30 13:10:42.826469 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.826439    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-podinfo\") pod \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\" (UID: \"237869eb-18ff-4f5a-87e8-e7d8c97bd088\") "
Oct 30 13:10:42.826960 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.826484    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-token\" (UniqueName: \"kubernetes.io/projected/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-token\") pod \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") "
Oct 30 13:10:42.826960 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.826515    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-podinfo\") pod \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") "
Oct 30 13:10:42.833843 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.825873    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-workload-socket" pod "b87eaba7-33de-46f3-a25d-2e86f1e12abe" (UID: "b87eaba7-33de-46f3-a25d-2e86f1e12abe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.833843 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.826513    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-workload-certs" pod "237869eb-18ff-4f5a-87e8-e7d8c97bd088" (UID: "237869eb-18ff-4f5a-87e8-e7d8c97bd088"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.833843 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.827016    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-data" pod "b87eaba7-33de-46f3-a25d-2e86f1e12abe" (UID: "b87eaba7-33de-46f3-a25d-2e86f1e12abe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.833843 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.827167    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-workload-certs" pod "b87eaba7-33de-46f3-a25d-2e86f1e12abe" (UID: "b87eaba7-33de-46f3-a25d-2e86f1e12abe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.833843 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.827399    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/473f256f-7ee3-4e49-b407-346e0242fcbb-istiod-ca-cert" pod "473f256f-7ee3-4e49-b407-346e0242fcbb" (UID: "473f256f-7ee3-4e49-b407-346e0242fcbb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:10:42.833843 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.827590    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-data" pod "237869eb-18ff-4f5a-87e8-e7d8c97bd088" (UID: "237869eb-18ff-4f5a-87e8-e7d8c97bd088"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.833843 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.829552    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-podinfo" pod "473f256f-7ee3-4e49-b407-346e0242fcbb" (UID: "473f256f-7ee3-4e49-b407-346e0242fcbb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:10:42.833843 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.831057    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-envoy" pod "237869eb-18ff-4f5a-87e8-e7d8c97bd088" (UID: "237869eb-18ff-4f5a-87e8-e7d8c97bd088"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.833843 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.831314    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-token" pod "237869eb-18ff-4f5a-87e8-e7d8c97bd088" (UID: "237869eb-18ff-4f5a-87e8-e7d8c97bd088"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:42.833843 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.831417    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/237869eb-18ff-4f5a-87e8-e7d8c97bd088-kube-api-access-kldn2" pod "237869eb-18ff-4f5a-87e8-e7d8c97bd088" (UID: "237869eb-18ff-4f5a-87e8-e7d8c97bd088"). InnerVolumeSpecName "kube-api-access-kldn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:42.833843 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.831529    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-workload-certs" pod "473f256f-7ee3-4e49-b407-346e0242fcbb" (UID: "473f256f-7ee3-4e49-b407-346e0242fcbb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.833843 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.831660    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-workload-socket" pod "473f256f-7ee3-4e49-b407-346e0242fcbb" (UID: "473f256f-7ee3-4e49-b407-346e0242fcbb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.833843 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.831750    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-data" pod "473f256f-7ee3-4e49-b407-346e0242fcbb" (UID: "473f256f-7ee3-4e49-b407-346e0242fcbb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.833843 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.832746    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/473f256f-7ee3-4e49-b407-346e0242fcbb-kube-api-access-dvm6v" pod "473f256f-7ee3-4e49-b407-346e0242fcbb" (UID: "473f256f-7ee3-4e49-b407-346e0242fcbb"). InnerVolumeSpecName "kube-api-access-dvm6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:42.833843 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.833377    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b87eaba7-33de-46f3-a25d-2e86f1e12abe-kube-api-access-r46dz" pod "b87eaba7-33de-46f3-a25d-2e86f1e12abe" (UID: "b87eaba7-33de-46f3-a25d-2e86f1e12abe"). InnerVolumeSpecName "kube-api-access-r46dz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:42.834722 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.833931    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istiod-ca-cert" pod "b87eaba7-33de-46f3-a25d-2e86f1e12abe" (UID: "b87eaba7-33de-46f3-a25d-2e86f1e12abe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:10:42.834722 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834150    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-envoy" pod "b87eaba7-33de-46f3-a25d-2e86f1e12abe" (UID: "b87eaba7-33de-46f3-a25d-2e86f1e12abe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.834722 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834170    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-credential-socket" pod "237869eb-18ff-4f5a-87e8-e7d8c97bd088" (UID: "237869eb-18ff-4f5a-87e8-e7d8c97bd088"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.834722 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834223    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-credential-socket\") pod \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\" (UID: \"b87eaba7-33de-46f3-a25d-2e86f1e12abe\") "
Oct 30 13:10:42.834722 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834231    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-envoy" pod "473f256f-7ee3-4e49-b407-346e0242fcbb" (UID: "473f256f-7ee3-4e49-b407-346e0242fcbb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.834722 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834442    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-workload-socket" pod "237869eb-18ff-4f5a-87e8-e7d8c97bd088" (UID: "237869eb-18ff-4f5a-87e8-e7d8c97bd088"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.834722 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834468    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istiod-ca-cert" pod "237869eb-18ff-4f5a-87e8-e7d8c97bd088" (UID: "237869eb-18ff-4f5a-87e8-e7d8c97bd088"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:10:42.834722 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834639    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-credential-socket" pod "b87eaba7-33de-46f3-a25d-2e86f1e12abe" (UID: "b87eaba7-33de-46f3-a25d-2e86f1e12abe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834761    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834780    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834796    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834810    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834824    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834838    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834853    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834866    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834880    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834894    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834910    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834925    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834939    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvm6v\" (UniqueName: \"kubernetes.io/projected/473f256f-7ee3-4e49-b407-346e0242fcbb-kube-api-access-dvm6v\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834955    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834970    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdbht\" (UniqueName: \"kubernetes.io/projected/26fbb81d-799f-47c7-97ab-c5a0e30425ac-kube-api-access-fdbht\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834985    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.834999    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835012    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26fbb81d-799f-47c7-97ab-c5a0e30425ac-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835025    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835041    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835054    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835070    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835084    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/473f256f-7ee3-4e49-b407-346e0242fcbb-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835098    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kldn2\" (UniqueName: \"kubernetes.io/projected/237869eb-18ff-4f5a-87e8-e7d8c97bd088-kube-api-access-kldn2\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835115    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r46dz\" (UniqueName: \"kubernetes.io/projected/b87eaba7-33de-46f3-a25d-2e86f1e12abe-kube-api-access-r46dz\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.835117 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835129    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.836277 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835144    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.836277 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835158    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.836277 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835172    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.836277 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835185    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/473f256f-7ee3-4e49-b407-346e0242fcbb-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.836277 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835199    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.836277 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835212    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26fbb81d-799f-47c7-97ab-c5a0e30425ac-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.836277 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.835478    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-podinfo" pod "237869eb-18ff-4f5a-87e8-e7d8c97bd088" (UID: "237869eb-18ff-4f5a-87e8-e7d8c97bd088"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:10:42.838488 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.838459    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-podinfo" pod "b87eaba7-33de-46f3-a25d-2e86f1e12abe" (UID: "b87eaba7-33de-46f3-a25d-2e86f1e12abe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:10:42.841818 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.841764    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-token" pod "b87eaba7-33de-46f3-a25d-2e86f1e12abe" (UID: "b87eaba7-33de-46f3-a25d-2e86f1e12abe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:42.844440 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.844410    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-token" pod "473f256f-7ee3-4e49-b407-346e0242fcbb" (UID: "473f256f-7ee3-4e49-b407-346e0242fcbb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:42.860227 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.860176    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 13:10:42.863171 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.863149    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 13:10:42.872453 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.872434    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-n6rkn_703d6675-c844-48eb-a019-2a09a2123307/manager/0.log"
Oct 30 13:10:42.935928 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.935898    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b107d489-5cd0-492b-8cab-01d603040f8b-kube-api-access-62thg\" (UniqueName: \"kubernetes.io/projected/b107d489-5cd0-492b-8cab-01d603040f8b-kube-api-access-62thg\") pod \"b107d489-5cd0-492b-8cab-01d603040f8b\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") "
Oct 30 13:10:42.935928 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.935936    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istiod-ca-cert\") pod \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") "
Oct 30 13:10:42.936194 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.935971    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-istio-envoy\") pod \"b107d489-5cd0-492b-8cab-01d603040f8b\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") "
Oct 30 13:10:42.936194 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936000    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-token\" (UniqueName: \"kubernetes.io/projected/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-token\") pod \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") "
Oct 30 13:10:42.936194 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936028    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b107d489-5cd0-492b-8cab-01d603040f8b-istio-token\" (UniqueName: \"kubernetes.io/projected/b107d489-5cd0-492b-8cab-01d603040f8b-istio-token\") pod \"b107d489-5cd0-492b-8cab-01d603040f8b\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") "
Oct 30 13:10:42.936194 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936056    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-workload-certs\") pod \"b107d489-5cd0-492b-8cab-01d603040f8b\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") "
Oct 30 13:10:42.936194 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936083    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-credential-socket\") pod \"b107d489-5cd0-492b-8cab-01d603040f8b\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") "
Oct 30 13:10:42.936194 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936114    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-istio-data\") pod \"b107d489-5cd0-492b-8cab-01d603040f8b\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") "
Oct 30 13:10:42.936194 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936156    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-kube-api-access-h6k8k\" (UniqueName: \"kubernetes.io/projected/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-kube-api-access-h6k8k\") pod \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") "
Oct 30 13:10:42.937026 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936204    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-workload-certs\") pod \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") "
Oct 30 13:10:42.937026 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936254    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b107d489-5cd0-492b-8cab-01d603040f8b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b107d489-5cd0-492b-8cab-01d603040f8b-istio-podinfo\") pod \"b107d489-5cd0-492b-8cab-01d603040f8b\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") "
Oct 30 13:10:42.937026 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936309    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-podinfo\") pod \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") "
Oct 30 13:10:42.937026 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936358    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-workload-socket\") pod \"b107d489-5cd0-492b-8cab-01d603040f8b\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") "
Oct 30 13:10:42.937026 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936422    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-data\") pod \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") "
Oct 30 13:10:42.937026 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936454    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b107d489-5cd0-492b-8cab-01d603040f8b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b107d489-5cd0-492b-8cab-01d603040f8b-istiod-ca-cert\") pod \"b107d489-5cd0-492b-8cab-01d603040f8b\" (UID: \"b107d489-5cd0-492b-8cab-01d603040f8b\") "
Oct 30 13:10:42.937026 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936496    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-envoy\") pod \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") "
Oct 30 13:10:42.937026 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936497    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-credential-socket" pod "b107d489-5cd0-492b-8cab-01d603040f8b" (UID: "b107d489-5cd0-492b-8cab-01d603040f8b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.937026 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936725    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-workload-certs" pod "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" (UID: "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.937026 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936762    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-credential-socket" pod "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" (UID: "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.937869 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.937253    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istiod-ca-cert" pod "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" (UID: "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:10:42.937869 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.937368    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-istio-data" pod "b107d489-5cd0-492b-8cab-01d603040f8b" (UID: "b107d489-5cd0-492b-8cab-01d603040f8b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.937869 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.937623    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-workload-certs" pod "b107d489-5cd0-492b-8cab-01d603040f8b" (UID: "b107d489-5cd0-492b-8cab-01d603040f8b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.937869 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.937650    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-data" pod "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" (UID: "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.938081 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.937962    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b107d489-5cd0-492b-8cab-01d603040f8b-istiod-ca-cert" pod "b107d489-5cd0-492b-8cab-01d603040f8b" (UID: "b107d489-5cd0-492b-8cab-01d603040f8b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 13:10:42.938151 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.936555    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-credential-socket\") pod \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") "
Oct 30 13:10:42.938223 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.938172    4945 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-workload-socket\") pod \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\" (UID: \"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3\") "
Oct 30 13:10:42.939189 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.939165    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-workload-socket" pod "b107d489-5cd0-492b-8cab-01d603040f8b" (UID: "b107d489-5cd0-492b-8cab-01d603040f8b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.939441 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.939422    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/473f256f-7ee3-4e49-b407-346e0242fcbb-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.939531 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.939445    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.939531 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.939459    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b107d489-5cd0-492b-8cab-01d603040f8b-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.939531 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.939471    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.939531 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.939484    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/237869eb-18ff-4f5a-87e8-e7d8c97bd088-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.939531 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.939498    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.939531 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.939511    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b87eaba7-33de-46f3-a25d-2e86f1e12abe-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.939531 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.939524    4945 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istiod-ca-cert\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.939946 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.939536    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.939946 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.939548    4945 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-credential-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.939946 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.939561    4945 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-istio-data\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.939946 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.939575    4945 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-workload-certs\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.939946 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.939588    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:42.939946 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.939864    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-workload-socket" pod "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" (UID: "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.945189 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.945141    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b107d489-5cd0-492b-8cab-01d603040f8b-istio-token" pod "b107d489-5cd0-492b-8cab-01d603040f8b" (UID: "b107d489-5cd0-492b-8cab-01d603040f8b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:42.945189 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.945142    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b107d489-5cd0-492b-8cab-01d603040f8b-kube-api-access-62thg" pod "b107d489-5cd0-492b-8cab-01d603040f8b" (UID: "b107d489-5cd0-492b-8cab-01d603040f8b"). InnerVolumeSpecName "kube-api-access-62thg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:42.945189 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.945163    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-kube-api-access-h6k8k" pod "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" (UID: "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3"). InnerVolumeSpecName "kube-api-access-h6k8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:42.945189 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.945137    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-podinfo" pod "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" (UID: "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:10:42.945189 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.945168    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b107d489-5cd0-492b-8cab-01d603040f8b-istio-podinfo" pod "b107d489-5cd0-492b-8cab-01d603040f8b" (UID: "b107d489-5cd0-492b-8cab-01d603040f8b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 13:10:42.945403 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.945277    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-token" pod "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" (UID: "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 13:10:42.945403 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.945358    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-envoy" pod "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" (UID: "492c3c1e-1d46-4e6f-9203-3d0a478fe8b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:42.945472 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:42.945444    4945 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-istio-envoy" pod "b107d489-5cd0-492b-8cab-01d603040f8b" (UID: "b107d489-5cd0-492b-8cab-01d603040f8b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 13:10:43.040634 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.040541    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b107d489-5cd0-492b-8cab-01d603040f8b-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:43.040634 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.040575    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:43.040634 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.040587    4945 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b107d489-5cd0-492b-8cab-01d603040f8b-istio-token\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:43.040634 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.040629    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6k8k\" (UniqueName: \"kubernetes.io/projected/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-kube-api-access-h6k8k\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:43.040914 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.040644    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b107d489-5cd0-492b-8cab-01d603040f8b-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:43.040914 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.040658    4945 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-podinfo\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:43.040914 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.040669    4945 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-istio-envoy\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:43.040914 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.040679    4945 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3-workload-socket\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:43.040914 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.040690    4945 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62thg\" (UniqueName: \"kubernetes.io/projected/b107d489-5cd0-492b-8cab-01d603040f8b-kube-api-access-62thg\") on node \"ip-10-0-142-36.ec2.internal\" DevicePath \"\""
Oct 30 13:10:43.127162 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.127124    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-ktflf_5745d316-11ab-412b-8e86-e3068d830cb7/proxy/0.log"
Oct 30 13:10:43.477153 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.477119    4945 generic.go:356] "Generic (PLEG): container finished" podID="492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" containerID="c0341fe52dcfb8e1d994335f8acfd2da7b613f0e3ea4711a740fb11d93216843" exitCode=0
Oct 30 13:10:43.477337 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.477201    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk" event={"ID":"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3","Type":"ContainerDied","Data":"c0341fe52dcfb8e1d994335f8acfd2da7b613f0e3ea4711a740fb11d93216843"}
Oct 30 13:10:43.477337 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.477289    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk" event={"ID":"492c3c1e-1d46-4e6f-9203-3d0a478fe8b3","Type":"ContainerDied","Data":"0d3aa12477f7d5de3cbc955e1ca7f4a7909e2b87bc069c51fa68ced774adb50f"}
Oct 30 13:10:43.477337 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.477318    4945 scope.go:122] "RemoveContainer" containerID="c0341fe52dcfb8e1d994335f8acfd2da7b613f0e3ea4711a740fb11d93216843"
Oct 30 13:10:43.477505 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.477481    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"
Oct 30 13:10:43.501786 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.497512    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th" event={"ID":"26fbb81d-799f-47c7-97ab-c5a0e30425ac","Type":"ContainerDied","Data":"cd91ea90d4b8155b139befc1077798b56a6dd2ad459469305a102462bbcc2733"}
Oct 30 13:10:43.501786 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.497679    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"
Oct 30 13:10:43.501786 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.501555    4945 generic.go:356] "Generic (PLEG): container finished" podID="b107d489-5cd0-492b-8cab-01d603040f8b" containerID="63b413b7ea2f4104826307b87408e1df9b9589795c2bec913e508474c4db6863" exitCode=0
Oct 30 13:10:43.501786 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.501664    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd" event={"ID":"b107d489-5cd0-492b-8cab-01d603040f8b","Type":"ContainerDied","Data":"63b413b7ea2f4104826307b87408e1df9b9589795c2bec913e508474c4db6863"}
Oct 30 13:10:43.501786 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.501760    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"
Oct 30 13:10:43.509003 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.508974    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd" event={"ID":"b107d489-5cd0-492b-8cab-01d603040f8b","Type":"ContainerDied","Data":"599366c5e97d9c39d2079a82516536c743870e54bdf95394c09fdedb4559cc79"}
Oct 30 13:10:43.516071 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.516050    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"
Oct 30 13:10:43.516195 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.516052    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r" event={"ID":"473f256f-7ee3-4e49-b407-346e0242fcbb","Type":"ContainerDied","Data":"8e535f5c0556d744b166261960389f479f3558bb38ca5453e30a4bf6df84fb63"}
Oct 30 13:10:43.522619 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.522577    4945 generic.go:356] "Generic (PLEG): container finished" podID="237869eb-18ff-4f5a-87e8-e7d8c97bd088" containerID="8185dd7b9cc870ecf6a1ad499fec7157df1842ea6f9aa7f81f2b79ca5388f322" exitCode=0
Oct 30 13:10:43.522891 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.522864    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9" event={"ID":"237869eb-18ff-4f5a-87e8-e7d8c97bd088","Type":"ContainerDied","Data":"8185dd7b9cc870ecf6a1ad499fec7157df1842ea6f9aa7f81f2b79ca5388f322"}
Oct 30 13:10:43.523004 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.522918    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9" event={"ID":"237869eb-18ff-4f5a-87e8-e7d8c97bd088","Type":"ContainerDied","Data":"8c554a49490cd6eb0ab42bd87faf72015b674091e9dddf36e3f8fe890fd4fedd"}
Oct 30 13:10:43.523188 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.523147    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"
Oct 30 13:10:43.527760 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.527740    4945 scope.go:122] "RemoveContainer" containerID="c0341fe52dcfb8e1d994335f8acfd2da7b613f0e3ea4711a740fb11d93216843"
Oct 30 13:10:43.528393 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:10:43.528358    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0341fe52dcfb8e1d994335f8acfd2da7b613f0e3ea4711a740fb11d93216843\": container with ID starting with c0341fe52dcfb8e1d994335f8acfd2da7b613f0e3ea4711a740fb11d93216843 not found: ID does not exist" containerID="c0341fe52dcfb8e1d994335f8acfd2da7b613f0e3ea4711a740fb11d93216843"
Oct 30 13:10:43.528494 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.528406    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0341fe52dcfb8e1d994335f8acfd2da7b613f0e3ea4711a740fb11d93216843"} err="failed to get container status \"c0341fe52dcfb8e1d994335f8acfd2da7b613f0e3ea4711a740fb11d93216843\": rpc error: code = NotFound desc = could not find container \"c0341fe52dcfb8e1d994335f8acfd2da7b613f0e3ea4711a740fb11d93216843\": container with ID starting with c0341fe52dcfb8e1d994335f8acfd2da7b613f0e3ea4711a740fb11d93216843 not found: ID does not exist"
Oct 30 13:10:43.528494 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.528432    4945 scope.go:122] "RemoveContainer" containerID="f64f4e18b1dae7ea1caa7e05a2cb5bdc0379308b1318b00d0ca1c6d4abbc64b7"
Oct 30 13:10:43.532083 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.532047    4945 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"
Oct 30 13:10:43.532271 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.532048    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw" event={"ID":"b87eaba7-33de-46f3-a25d-2e86f1e12abe","Type":"ContainerDied","Data":"fc64f3e3ad6b96edb9937b3cb9e751f3cb16375a13b7eac0e26c1b3a79a09313"}
Oct 30 13:10:43.535450 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.535377    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/0.log"
Oct 30 13:10:43.568944 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.568920    4945 scope.go:122] "RemoveContainer" containerID="63b413b7ea2f4104826307b87408e1df9b9589795c2bec913e508474c4db6863"
Oct 30 13:10:43.581397 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.581369    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"]
Oct 30 13:10:43.582479 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.582362    4945 scope.go:122] "RemoveContainer" containerID="63b413b7ea2f4104826307b87408e1df9b9589795c2bec913e508474c4db6863"
Oct 30 13:10:43.583101 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:10:43.582936    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63b413b7ea2f4104826307b87408e1df9b9589795c2bec913e508474c4db6863\": container with ID starting with 63b413b7ea2f4104826307b87408e1df9b9589795c2bec913e508474c4db6863 not found: ID does not exist" containerID="63b413b7ea2f4104826307b87408e1df9b9589795c2bec913e508474c4db6863"
Oct 30 13:10:43.583101 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.582993    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63b413b7ea2f4104826307b87408e1df9b9589795c2bec913e508474c4db6863"} err="failed to get container status \"63b413b7ea2f4104826307b87408e1df9b9589795c2bec913e508474c4db6863\": rpc error: code = NotFound desc = could not find container \"63b413b7ea2f4104826307b87408e1df9b9589795c2bec913e508474c4db6863\": container with ID starting with 63b413b7ea2f4104826307b87408e1df9b9589795c2bec913e508474c4db6863 not found: ID does not exist"
Oct 30 13:10:43.583101 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.583019    4945 scope.go:122] "RemoveContainer" containerID="8fbbe50b7d48c9089db28460562410ad41175fd9c24f0c5e69166d2dbd686a73"
Oct 30 13:10:43.585322 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.585297    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-cg9mk"]
Oct 30 13:10:43.595792 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.595766    4945 scope.go:122] "RemoveContainer" containerID="8185dd7b9cc870ecf6a1ad499fec7157df1842ea6f9aa7f81f2b79ca5388f322"
Oct 30 13:10:43.606571 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.606551    4945 scope.go:122] "RemoveContainer" containerID="8185dd7b9cc870ecf6a1ad499fec7157df1842ea6f9aa7f81f2b79ca5388f322"
Oct 30 13:10:43.606987 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:10:43.606954    4945 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8185dd7b9cc870ecf6a1ad499fec7157df1842ea6f9aa7f81f2b79ca5388f322\": container with ID starting with 8185dd7b9cc870ecf6a1ad499fec7157df1842ea6f9aa7f81f2b79ca5388f322 not found: ID does not exist" containerID="8185dd7b9cc870ecf6a1ad499fec7157df1842ea6f9aa7f81f2b79ca5388f322"
Oct 30 13:10:43.607056 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.606989    4945 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8185dd7b9cc870ecf6a1ad499fec7157df1842ea6f9aa7f81f2b79ca5388f322"} err="failed to get container status \"8185dd7b9cc870ecf6a1ad499fec7157df1842ea6f9aa7f81f2b79ca5388f322\": rpc error: code = NotFound desc = could not find container \"8185dd7b9cc870ecf6a1ad499fec7157df1842ea6f9aa7f81f2b79ca5388f322\": container with ID starting with 8185dd7b9cc870ecf6a1ad499fec7157df1842ea6f9aa7f81f2b79ca5388f322 not found: ID does not exist"
Oct 30 13:10:43.607056 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.607010    4945 scope.go:122] "RemoveContainer" containerID="942bad7daf144f602f87b78beb1b99bfeda4ebb86ae9d17a901c1dfa6ab2b6e5"
Oct 30 13:10:43.618920 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.618898    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"]
Oct 30 13:10:43.632999 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.632964    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-nn8th"]
Oct 30 13:10:43.676817 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.676784    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"]
Oct 30 13:10:43.680943 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.680917    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-2kshw"]
Oct 30 13:10:43.716931 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.716895    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"]
Oct 30 13:10:43.721113 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.721039    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-53dcb1-openshift-default-5d7f9b4f49-fwk8r"]
Oct 30 13:10:43.756206 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.756174    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"]
Oct 30 13:10:43.759990 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.759964    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-9gtr9"]
Oct 30 13:10:43.793852 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.793746    4945 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"]
Oct 30 13:10:43.796436 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:43.796386    4945 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-7a590a-openshift-default-7d55f4c4df-nwmdd"]
Oct 30 13:10:45.552963 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:45.552928    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="237869eb-18ff-4f5a-87e8-e7d8c97bd088" path="/var/lib/kubelet/pods/237869eb-18ff-4f5a-87e8-e7d8c97bd088/volumes"
Oct 30 13:10:45.553356 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:45.553341    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" path="/var/lib/kubelet/pods/26fbb81d-799f-47c7-97ab-c5a0e30425ac/volumes"
Oct 30 13:10:45.553790 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:45.553774    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="473f256f-7ee3-4e49-b407-346e0242fcbb" path="/var/lib/kubelet/pods/473f256f-7ee3-4e49-b407-346e0242fcbb/volumes"
Oct 30 13:10:45.554637 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:45.554593    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" path="/var/lib/kubelet/pods/492c3c1e-1d46-4e6f-9203-3d0a478fe8b3/volumes"
Oct 30 13:10:45.555057 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:45.555042    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b107d489-5cd0-492b-8cab-01d603040f8b" path="/var/lib/kubelet/pods/b107d489-5cd0-492b-8cab-01d603040f8b/volumes"
Oct 30 13:10:45.555546 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:45.555520    4945 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b87eaba7-33de-46f3-a25d-2e86f1e12abe" path="/var/lib/kubelet/pods/b87eaba7-33de-46f3-a25d-2e86f1e12abe/volumes"
Oct 30 13:10:54.649798 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:54.649767    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-5k947_55710abf-0a0a-4830-a4dd-54083ab2ac04/manager/0.log"
Oct 30 13:10:54.841637 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:54.841584    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-s9qks_85cd2a7e-fce3-43f2-8d41-feaf9406baa5/manager/0.log"
Oct 30 13:10:55.051794 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:55.051761    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-z228w_fbcc3687-0c22-414a-aacc-5f18a80f7d41/manager/0.log"
Oct 30 13:10:55.510212 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:55.510164    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/1.log"
Oct 30 13:10:55.678648 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:55.678618    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gzpff_b818f62c-1645-4782-93a6-da2341cdcce7/server/0.log"
Oct 30 13:10:55.867340 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:55.867267    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-n6rkn_703d6675-c844-48eb-a019-2a09a2123307/manager/0.log"
Oct 30 13:10:56.088612 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:56.088577    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-ktflf_5745d316-11ab-412b-8e86-e3068d830cb7/proxy/0.log"
Oct 30 13:10:56.191335 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:10:56.191249    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:10:56.389269 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:56.389237    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 13:10:56.392436 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:56.392410    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 13:10:56.490940 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:56.490900    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/0.log"
Oct 30 13:10:57.021479 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:57.021452    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8664b8665c-7s9c8_cb36a502-be60-4114-9d7b-09cbdab505f5/maas-api/0.log"
Oct 30 13:10:59.579352 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:10:59.579316    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-cnwvz_8e1934f7-aafd-4e61-8f49-4a530f77403d/postgres/0.log"
Oct 30 13:11:01.229948 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:01.229911    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-779bff956b-l69xd_f6d044ab-287d-434c-8fd5-bb9b61eaffaa/manager/0.log"
Oct 30 13:11:01.411502 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:01.411476    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_developer-portal-controller-66d8dbb8c8-jxp7q_2b240210-5d58-4791-8745-60c58f3e9bbf/manager/0.log"
Oct 30 13:11:01.611460 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:01.611369    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-57f5c66455-9g9vc_e0821ac0-2e9b-4403-914a-e13b7130e8a2/manager/0.log"
Oct 30 13:11:01.951257 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:01.951231    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6b575997cc-7g9t5_607ad48d-3968-4ffd-973f-51272eb00937/manager/0.log"
Oct 30 13:11:02.507027 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:02.507000    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_mcp-gateway-controller-6c98b995c5-j98z2_a59f98de-82e6-4a91-bdab-beb39922f5ff/mcp-controller/0.log"
Oct 30 13:11:03.209180 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:03.209146    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-858f55f4d9t74ft_b1c8ceec-1c52-4523-ae41-69be10ac654d/istio-proxy/0.log"
Oct 30 13:11:03.403540 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:03.403506    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-6jqvg_bfbdeeba-d2f7-4b98-96f6-6136e0c79621/discovery/0.log"
Oct 30 13:11:04.381168 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:04.381140    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-5fwdk_c8885359-7cb6-4e2e-89ca-c1499f813ea7/istio-proxy/0.log"
Oct 30 13:11:04.949023 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:04.948990    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5_6e9dc79c-741f-465e-ba5d-3c99e6d7f812/istio-proxy/0.log"
Oct 30 13:11:05.525961 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:05.525926    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-846cf5b65-gllvj_d81730d3-53ed-40b8-a512-348fc042f821/router/0.log"
Oct 30 13:11:24.322618 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:24.322529    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ntzbv_15bb3274-f845-4851-90ae-f4ab08e685c8/global-pull-secret-syncer/0.log"
Oct 30 13:11:24.475246 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:24.475219    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-5lwxr_e30d5797-e034-4806-be4f-54f54698920c/connection-checker/0.log"
Oct 30 13:11:24.585210 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:24.585135    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-q74lv_06910518-58db-450f-a5ee-521ac8f51e1a/connection-checker/0.log"
Oct 30 13:11:24.691898 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:24.691873    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-q8mz6_79f0f110-a04f-4b35-8eba-c8f0469b62a1/connection-checker/0.log"
Oct 30 13:11:25.051530 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:25.051496    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-p2z49_7d7b7ac0-cec8-4b06-8bb9-ef5a458f42ce/konnectivity-agent/0.log"
Oct 30 13:11:25.390359 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:25.390281    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-36.ec2.internal_6766e4bd763f65c3c761bb65d5e5c301/haproxy/0.log"
Oct 30 13:11:26.190684 ip-10-0-142-36 kubenswrapper[4945]: E1030 13:11:26.190648    4945 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-lw4gw" containerName="extractor"
Oct 30 13:11:32.205297 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:32.205264    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-779bff956b-l69xd_f6d044ab-287d-434c-8fd5-bb9b61eaffaa/manager/0.log"
Oct 30 13:11:32.479145 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:32.479055    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_developer-portal-controller-66d8dbb8c8-jxp7q_2b240210-5d58-4791-8745-60c58f3e9bbf/manager/0.log"
Oct 30 13:11:32.592583 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:32.592555    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-57f5c66455-9g9vc_e0821ac0-2e9b-4403-914a-e13b7130e8a2/manager/0.log"
Oct 30 13:11:32.843318 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:32.843238    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6b575997cc-7g9t5_607ad48d-3968-4ffd-973f-51272eb00937/manager/0.log"
Oct 30 13:11:33.892373 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:33.892340    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_mcp-gateway-controller-6c98b995c5-j98z2_a59f98de-82e6-4a91-bdab-beb39922f5ff/mcp-controller/0.log"
Oct 30 13:11:35.297076 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297039    4945 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"]
Oct 30 13:11:35.297483 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297449    4945 cpu_manager.go:414] "containerMap: removing container" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy"
Oct 30 13:11:35.297483 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297460    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy"
Oct 30 13:11:35.297483 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297476    4945 cpu_manager.go:414] "containerMap: removing container" podUID="bae96c60-5094-420b-8dd6-8bba39c588a8" containerName="maas-api"
Oct 30 13:11:35.297483 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297481    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bae96c60-5094-420b-8dd6-8bba39c588a8" containerName="maas-api"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297509    4945 cpu_manager.go:414] "containerMap: removing container" podUID="492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297514    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297525    4945 cpu_manager.go:414] "containerMap: removing container" podUID="473f256f-7ee3-4e49-b407-346e0242fcbb" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297530    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="473f256f-7ee3-4e49-b407-346e0242fcbb" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297537    4945 cpu_manager.go:414] "containerMap: removing container" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297541    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297548    4945 cpu_manager.go:414] "containerMap: removing container" podUID="b107d489-5cd0-492b-8cab-01d603040f8b" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297553    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b107d489-5cd0-492b-8cab-01d603040f8b" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297559    4945 cpu_manager.go:414] "containerMap: removing container" podUID="b87eaba7-33de-46f3-a25d-2e86f1e12abe" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297563    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b87eaba7-33de-46f3-a25d-2e86f1e12abe" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297573    4945 cpu_manager.go:414] "containerMap: removing container" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297577    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297584    4945 cpu_manager.go:414] "containerMap: removing container" podUID="237869eb-18ff-4f5a-87e8-e7d8c97bd088" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297588    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="237869eb-18ff-4f5a-87e8-e7d8c97bd088" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297610    4945 cpu_manager.go:414] "containerMap: removing container" podUID="042f8c8d-a857-4efb-b8b5-6681e172fe1e" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297614    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="042f8c8d-a857-4efb-b8b5-6681e172fe1e" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297624    4945 cpu_manager.go:414] "containerMap: removing container" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297628    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297634    4945 cpu_manager.go:414] "containerMap: removing container" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297639    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerName="istio-proxy"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297647    4945 cpu_manager.go:414] "containerMap: removing container" podUID="a6ade70c-d9ac-4854-baae-0190ebd5ae08" containerName="maas-api"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297651    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6ade70c-d9ac-4854-baae-0190ebd5ae08" containerName="maas-api"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297660    4945 cpu_manager.go:414] "containerMap: removing container" podUID="579928aa-fcf1-4385-95ac-5d8f8648e5be" containerName="maas-api"
Oct 30 13:11:35.297659 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297664    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="579928aa-fcf1-4385-95ac-5d8f8648e5be" containerName="maas-api"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297673    4945 cpu_manager.go:414] "containerMap: removing container" podUID="15f559f9-e4fa-43f3-8308-a0c9a947080c" containerName="maas-api"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297678    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="15f559f9-e4fa-43f3-8308-a0c9a947080c" containerName="maas-api"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297684    4945 cpu_manager.go:414] "containerMap: removing container" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297688    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297695    4945 cpu_manager.go:414] "containerMap: removing container" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297699    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297707    4945 cpu_manager.go:414] "containerMap: removing container" podUID="951278f9-b9f4-43bd-920c-fb92e3fbed25" containerName="istio-proxy"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297711    4945 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="951278f9-b9f4-43bd-920c-fb92e3fbed25" containerName="istio-proxy"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297770    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="b107d489-5cd0-492b-8cab-01d603040f8b" containerName="istio-proxy"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297780    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="b87eaba7-33de-46f3-a25d-2e86f1e12abe" containerName="istio-proxy"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297787    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="4fb24330-4337-4c41-80d0-e780d94119a4" containerName="istio-proxy"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297792    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="237869eb-18ff-4f5a-87e8-e7d8c97bd088" containerName="istio-proxy"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297797    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="042f8c8d-a857-4efb-b8b5-6681e172fe1e" containerName="istio-proxy"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297803    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0ba9b13-4372-4d8f-9916-30246cc34e84" containerName="istio-proxy"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297809    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="26fbb81d-799f-47c7-97ab-c5a0e30425ac" containerName="istio-proxy"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297814    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="a6ade70c-d9ac-4854-baae-0190ebd5ae08" containerName="maas-api"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297820    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="579928aa-fcf1-4385-95ac-5d8f8648e5be" containerName="maas-api"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297825    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="15f559f9-e4fa-43f3-8308-a0c9a947080c" containerName="maas-api"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297830    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a1454c3-5b47-4a17-8dd9-dc9ba7735caa" containerName="main"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297836    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a699888-47fc-44b7-88fb-26b4a0efecba" containerName="main"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297841    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="951278f9-b9f4-43bd-920c-fb92e3fbed25" containerName="istio-proxy"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297846    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3611f91-0fbc-47ff-b790-d635fb9d2e21" containerName="istio-proxy"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297851    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="bae96c60-5094-420b-8dd6-8bba39c588a8" containerName="maas-api"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297859    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="492c3c1e-1d46-4e6f-9203-3d0a478fe8b3" containerName="istio-proxy"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297865    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="473f256f-7ee3-4e49-b407-346e0242fcbb" containerName="istio-proxy"
Oct 30 13:11:35.298404 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.297869    4945 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3bc8355-ae84-4811-935f-0046ec42375e" containerName="main"
Oct 30 13:11:35.301199 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.301180    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.321311 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.321287    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"]
Oct 30 13:11:35.331534 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.331511    4945 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5tjtq\"/\"default-dockercfg-rwqwj\""
Oct 30 13:11:35.350924 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.350895    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b45ca74d-9d96-4569-af49-59f5d3ba5a44-proc\") pod \"perf-node-gather-daemonset-flt5m\" (UID: \"b45ca74d-9d96-4569-af49-59f5d3ba5a44\") " pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.351024 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.350935    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b45ca74d-9d96-4569-af49-59f5d3ba5a44-sys\") pod \"perf-node-gather-daemonset-flt5m\" (UID: \"b45ca74d-9d96-4569-af49-59f5d3ba5a44\") " pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.351024 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.350995    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6fkg\" (UniqueName: \"kubernetes.io/projected/b45ca74d-9d96-4569-af49-59f5d3ba5a44-kube-api-access-x6fkg\") pod \"perf-node-gather-daemonset-flt5m\" (UID: \"b45ca74d-9d96-4569-af49-59f5d3ba5a44\") " pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.351024 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.351019    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b45ca74d-9d96-4569-af49-59f5d3ba5a44-podres\") pod \"perf-node-gather-daemonset-flt5m\" (UID: \"b45ca74d-9d96-4569-af49-59f5d3ba5a44\") " pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.351130 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.351046    4945 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b45ca74d-9d96-4569-af49-59f5d3ba5a44-lib-modules\") pod \"perf-node-gather-daemonset-flt5m\" (UID: \"b45ca74d-9d96-4569-af49-59f5d3ba5a44\") " pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.408918 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.408884    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5tjtq\"/\"openshift-service-ca.crt\""
Oct 30 13:11:35.409071 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.408981    4945 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5tjtq\"/\"kube-root-ca.crt\""
Oct 30 13:11:35.452180 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.452145    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6fkg\" (UniqueName: \"kubernetes.io/projected/b45ca74d-9d96-4569-af49-59f5d3ba5a44-kube-api-access-x6fkg\") pod \"perf-node-gather-daemonset-flt5m\" (UID: \"b45ca74d-9d96-4569-af49-59f5d3ba5a44\") " pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.452351 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.452184    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b45ca74d-9d96-4569-af49-59f5d3ba5a44-podres\") pod \"perf-node-gather-daemonset-flt5m\" (UID: \"b45ca74d-9d96-4569-af49-59f5d3ba5a44\") " pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.452351 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.452213    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b45ca74d-9d96-4569-af49-59f5d3ba5a44-lib-modules\") pod \"perf-node-gather-daemonset-flt5m\" (UID: \"b45ca74d-9d96-4569-af49-59f5d3ba5a44\") " pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.452351 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.452281    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b45ca74d-9d96-4569-af49-59f5d3ba5a44-proc\") pod \"perf-node-gather-daemonset-flt5m\" (UID: \"b45ca74d-9d96-4569-af49-59f5d3ba5a44\") " pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.452351 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.452297    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b45ca74d-9d96-4569-af49-59f5d3ba5a44-podres\") pod \"perf-node-gather-daemonset-flt5m\" (UID: \"b45ca74d-9d96-4569-af49-59f5d3ba5a44\") " pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.452351 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.452306    4945 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b45ca74d-9d96-4569-af49-59f5d3ba5a44-sys\") pod \"perf-node-gather-daemonset-flt5m\" (UID: \"b45ca74d-9d96-4569-af49-59f5d3ba5a44\") " pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.452351 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.452347    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b45ca74d-9d96-4569-af49-59f5d3ba5a44-sys\") pod \"perf-node-gather-daemonset-flt5m\" (UID: \"b45ca74d-9d96-4569-af49-59f5d3ba5a44\") " pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.452571 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.452361    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b45ca74d-9d96-4569-af49-59f5d3ba5a44-lib-modules\") pod \"perf-node-gather-daemonset-flt5m\" (UID: \"b45ca74d-9d96-4569-af49-59f5d3ba5a44\") " pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.452571 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.452370    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b45ca74d-9d96-4569-af49-59f5d3ba5a44-proc\") pod \"perf-node-gather-daemonset-flt5m\" (UID: \"b45ca74d-9d96-4569-af49-59f5d3ba5a44\") " pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.480930 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.480899    4945 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6fkg\" (UniqueName: \"kubernetes.io/projected/b45ca74d-9d96-4569-af49-59f5d3ba5a44-kube-api-access-x6fkg\") pod \"perf-node-gather-daemonset-flt5m\" (UID: \"b45ca74d-9d96-4569-af49-59f5d3ba5a44\") " pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.560040 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.559959    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-5k947_55710abf-0a0a-4830-a4dd-54083ab2ac04/manager/0.log"
Oct 30 13:11:35.614689 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.614653    4945 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:35.656858 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.656829    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-5k947_55710abf-0a0a-4830-a4dd-54083ab2ac04/manager/0.log"
Oct 30 13:11:35.660127 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.660103    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-s9qks_85cd2a7e-fce3-43f2-8d41-feaf9406baa5/manager/0.log"
Oct 30 13:11:35.776331 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.776298    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-s9qks_85cd2a7e-fce3-43f2-8d41-feaf9406baa5/manager/0.log"
Oct 30 13:11:35.923414 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:35.923380    4945 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"]
Oct 30 13:11:36.464157 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:36.464115    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-z228w_fbcc3687-0c22-414a-aacc-5f18a80f7d41/manager/0.log"
Oct 30 13:11:36.488878 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:36.488843    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-48t4k_3a2f595c-6efe-4b69-81bf-074985a6ddde/cluster-monitoring-operator/0.log"
Oct 30 13:11:36.515973 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:36.515939    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/0.log"
Oct 30 13:11:36.615256 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:36.615219    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-z228w_fbcc3687-0c22-414a-aacc-5f18a80f7d41/manager/0.log"
Oct 30 13:11:36.665802 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:36.665767    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/0.log"
Oct 30 13:11:36.833692 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:36.833583    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m" event={"ID":"b45ca74d-9d96-4569-af49-59f5d3ba5a44","Type":"ContainerStarted","Data":"2efad7fc1f2415cdce199f2f1fd923458da3b9822ff82aafe8ef04c391f299e2"}
Oct 30 13:11:36.833849 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:36.833701    4945 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m" event={"ID":"b45ca74d-9d96-4569-af49-59f5d3ba5a44","Type":"ContainerStarted","Data":"38922e9b9b9ff3551886523af620098ab355ad0aa6ec8ffa29fa65bd1e553c55"}
Oct 30 13:11:36.833939 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:36.833919    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:36.856520 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:36.856490    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/1.log"
Oct 30 13:11:36.874756 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:36.874699    4945 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m" podStartSLOduration=1.874683955 podStartE2EDuration="1.874683955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 13:11:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:36.873464646 +0000 UTC m=+2141.451413735" watchObservedRunningTime="2026-10-30 13:11:36.874683955 +0000 UTC m=+2141.452633043"
Oct 30 13:11:36.893190 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:36.893153    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gzpff_b818f62c-1645-4782-93a6-da2341cdcce7/server/0.log"
Oct 30 13:11:36.977715 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:36.977643    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/1.log"
Oct 30 13:11:37.068356 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:37.068325    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gzpff_b818f62c-1645-4782-93a6-da2341cdcce7/server/0.log"
Oct 30 13:11:37.975426 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:37.975389    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p9pcf_6a83a03a-d39a-4814-bfa0-735fdd994de6/node-exporter/0.log"
Oct 30 13:11:38.095120 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:38.095092    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p9pcf_6a83a03a-d39a-4814-bfa0-735fdd994de6/kube-rbac-proxy/0.log"
Oct 30 13:11:38.331542 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:38.331457    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p9pcf_6a83a03a-d39a-4814-bfa0-735fdd994de6/init-textfile/0.log"
Oct 30 13:11:39.524504 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:39.524470    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-n6rkn_703d6675-c844-48eb-a019-2a09a2123307/manager/0.log"
Oct 30 13:11:39.568016 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:39.567987    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-ktflf_5745d316-11ab-412b-8e86-e3068d830cb7/proxy/0.log"
Oct 30 13:11:39.690348 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:39.690310    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-n6rkn_703d6675-c844-48eb-a019-2a09a2123307/manager/0.log"
Oct 30 13:11:39.731951 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:39.731911    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-ktflf_5745d316-11ab-412b-8e86-e3068d830cb7/proxy/0.log"
Oct 30 13:11:41.028741 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:41.028698    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-5k947_55710abf-0a0a-4830-a4dd-54083ab2ac04/manager/0.log"
Oct 30 13:11:41.125803 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:41.125774    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-s9qks_85cd2a7e-fce3-43f2-8d41-feaf9406baa5/manager/0.log"
Oct 30 13:11:41.254740 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:41.254710    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-5k947_55710abf-0a0a-4830-a4dd-54083ab2ac04/manager/0.log"
Oct 30 13:11:41.372053 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:41.371960    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-s9qks_85cd2a7e-fce3-43f2-8d41-feaf9406baa5/manager/0.log"
Oct 30 13:11:41.847584 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:41.847546    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-z228w_fbcc3687-0c22-414a-aacc-5f18a80f7d41/manager/0.log"
Oct 30 13:11:41.905511 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:41.905478    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/0.log"
Oct 30 13:11:42.159700 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:42.159619    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-z228w_fbcc3687-0c22-414a-aacc-5f18a80f7d41/manager/0.log"
Oct 30 13:11:42.221279 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:42.221246    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/0.log"
Oct 30 13:11:42.222673 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:42.222650    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/1.log"
Oct 30 13:11:42.274201 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:42.274166    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gzpff_b818f62c-1645-4782-93a6-da2341cdcce7/server/0.log"
Oct 30 13:11:42.516883 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:42.516833    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/1.log"
Oct 30 13:11:42.562266 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:42.562241    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gzpff_b818f62c-1645-4782-93a6-da2341cdcce7/server/0.log"
Oct 30 13:11:42.755444 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:42.755310    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-44v66_49837f84-8643-4643-aa92-3716a9b755d1/networking-console-plugin/0.log"
Oct 30 13:11:42.853693 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:42.853594    4945 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5tjtq/perf-node-gather-daemonset-flt5m"
Oct 30 13:11:44.004439 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:44.004390    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/2.log"
Oct 30 13:11:44.019416 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:44.019383    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-8gvm8_b80ffbfe-445f-431c-b8f4-be9fd2c5d094/console-operator/3.log"
Oct 30 13:11:44.941752 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:44.941716    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-n6rkn_703d6675-c844-48eb-a019-2a09a2123307/manager/0.log"
Oct 30 13:11:45.048355 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:45.048324    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-ktflf_5745d316-11ab-412b-8e86-e3068d830cb7/proxy/0.log"
Oct 30 13:11:45.186706 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:45.186676    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-n6rkn_703d6675-c844-48eb-a019-2a09a2123307/manager/0.log"
Oct 30 13:11:45.276685 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:45.276658    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-ktflf_5745d316-11ab-412b-8e86-e3068d830cb7/proxy/0.log"
Oct 30 13:11:46.892873 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:46.892841    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-5k947_55710abf-0a0a-4830-a4dd-54083ab2ac04/manager/0.log"
Oct 30 13:11:46.928380 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:46.928354    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-6mt4g_336de054-d674-4f26-81d8-980b11e27ace/volume-data-source-validator/0.log"
Oct 30 13:11:46.987101 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:46.987065    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-s9qks_85cd2a7e-fce3-43f2-8d41-feaf9406baa5/manager/0.log"
Oct 30 13:11:47.193496 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:47.193448    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-5k947_55710abf-0a0a-4830-a4dd-54083ab2ac04/manager/0.log"
Oct 30 13:11:47.307619 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:47.307553    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-s9qks_85cd2a7e-fce3-43f2-8d41-feaf9406baa5/manager/0.log"
Oct 30 13:11:47.654356 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:47.654259    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-z228w_fbcc3687-0c22-414a-aacc-5f18a80f7d41/manager/0.log"
Oct 30 13:11:47.721190 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:47.721152    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/0.log"
Oct 30 13:11:48.081125 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:48.081097    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/1.log"
Oct 30 13:11:48.109846 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:48.109808    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-z228w_fbcc3687-0c22-414a-aacc-5f18a80f7d41/manager/0.log"
Oct 30 13:11:48.123884 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:48.123858    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gzpff_b818f62c-1645-4782-93a6-da2341cdcce7/server/0.log"
Oct 30 13:11:48.182434 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:48.182400    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/0.log"
Oct 30 13:11:48.521610 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:48.521572    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d7d58c6ff-pncqs_180c4327-13ad-48b0-b5f1-c193ab9b0c9d/manager/1.log"
Oct 30 13:11:48.630824 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:48.630792    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gzpff_b818f62c-1645-4782-93a6-da2341cdcce7/server/0.log"
Oct 30 13:11:48.999121 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:48.999072    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-858f55f4d9t74ft_b1c8ceec-1c52-4523-ae41-69be10ac654d/istio-proxy/0.log"
Oct 30 13:11:49.504236 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:49.504203    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-6jqvg_bfbdeeba-d2f7-4b98-96f6-6136e0c79621/discovery/0.log"
Oct 30 13:11:49.556708 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:49.556674    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2djq9_e089d179-e583-4590-ba4c-39e7d486d707/dns/0.log"
Oct 30 13:11:49.686577 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:49.686546    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2djq9_e089d179-e583-4590-ba4c-39e7d486d707/kube-rbac-proxy/0.log"
Oct 30 13:11:50.718008 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:50.717976    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cwcj7_e014afb8-e7d0-4e32-97ff-dcba1bd2b8d0/dns-node-resolver/0.log"
Oct 30 13:11:51.021636 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:51.021511    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-n6rkn_703d6675-c844-48eb-a019-2a09a2123307/manager/0.log"
Oct 30 13:11:51.157037 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:51.157000    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-ktflf_5745d316-11ab-412b-8e86-e3068d830cb7/proxy/0.log"
Oct 30 13:11:51.449743 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:51.449707    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-n6rkn_703d6675-c844-48eb-a019-2a09a2123307/manager/0.log"
Oct 30 13:11:51.530961 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:51.530933    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-ktflf_5745d316-11ab-412b-8e86-e3068d830cb7/proxy/0.log"
Oct 30 13:11:52.531493 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:52.531450    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pt7kp_a594de5b-9748-4355-86b8-95ca7dc9dda1/node-ca/0.log"
Oct 30 13:11:53.141576 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:53.141541    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-858f55f4d9t74ft_b1c8ceec-1c52-4523-ae41-69be10ac654d/istio-proxy/0.log"
Oct 30 13:11:53.459580 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:53.459552    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-6jqvg_bfbdeeba-d2f7-4b98-96f6-6136e0c79621/discovery/0.log"
Oct 30 13:11:53.486825 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:53.486790    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-858f55f4d9t74ft_b1c8ceec-1c52-4523-ae41-69be10ac654d/istio-proxy/0.log"
Oct 30 13:11:53.853159 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:53.853084    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-6jqvg_bfbdeeba-d2f7-4b98-96f6-6136e0c79621/discovery/0.log"
Oct 30 13:11:54.391243 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:54.391208    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-5fwdk_c8885359-7cb6-4e2e-89ca-c1499f813ea7/istio-proxy/0.log"
Oct 30 13:11:54.590163 ip-10-0-142-36 kubenswrapper[4945]: I1030 13:11:54.590127    4945 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-6dd6bfb767-dwgn5_6e9dc79c-741f-465e-ba5d-3c99e6d7f812/istio-proxy/0.log"