Nov 03 20:36:30.865762 ip-10-0-142-223 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 03 20:36:30.865774 ip-10-0-142-223 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 03 20:36:30.865781 ip-10-0-142-223 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 03 20:36:30.866004 ip-10-0-142-223 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 03 20:36:40.912239 ip-10-0-142-223 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 03 20:36:40.912259 ip-10-0-142-223 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 26f56577987147c0863ec4eaf49187ee --
Nov 03 20:38:40.820330 ip-10-0-142-223 systemd[1]: Starting Kubernetes Kubelet...
Nov 03 20:38:41.292560 ip-10-0-142-223 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 20:38:41.292560 ip-10-0-142-223 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 03 20:38:41.292560 ip-10-0-142-223 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 20:38:41.292560 ip-10-0-142-223 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 20:38:41.292560 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198645    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:41.292560 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198714    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:41.292560 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198717    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:41.292560 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198720    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:41.292560 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198723    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:41.292560 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198726    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:41.292560 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198729    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:41.292560 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198731    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:41.292560 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198734    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:41.292560 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198736    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:41.292560 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198739    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198742    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198745    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198748    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198751    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198753    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198756    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198759    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198762    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198765    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198769    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198772    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198774    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198777    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198779    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198783    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198785    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198788    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198791    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198793    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:41.293865 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198796    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198799    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198801    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198804    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198806    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198809    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198811    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198814    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198816    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198819    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198822    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198825    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198827    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198830    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198832    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198835    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198837    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198844    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198847    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198849    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:41.294415 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198852    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198854    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198857    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198859    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198861    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198864    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198867    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198869    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198872    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198875    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198877    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198880    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198882    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198885    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198888    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198890    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198893    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198897    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198901    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:41.294918 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198904    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198906    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198909    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198912    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198914    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198917    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198919    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198922    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198926    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198930    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198934    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198937    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198940    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198943    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198945    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198948    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198950    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198953    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198955    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198958    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:41.352813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198961    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198963    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198966    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198968    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198971    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198973    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198975    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198979    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198982    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198984    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198987    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198989    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.198991    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199013    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199017    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199022    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199042    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199786    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199793    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:41.353674 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199796    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199799    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199801    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199804    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199806    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199810    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199812    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199815    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199818    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199820    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199823    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199825    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199828    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199831    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199834    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199836    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199839    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199841    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199845    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199847    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:41.354747 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199849    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199853    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199857    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199860    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199863    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199865    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199868    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199870    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199877    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199881    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199885    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199888    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199890    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199893    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199896    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199899    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199902    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199905    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199908    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:41.355872 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199910    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199913    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199916    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199918    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199921    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199923    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199926    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199928    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199931    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199934    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199936    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199939    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199941    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199944    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199947    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199949    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199951    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199954    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199956    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199959    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:41.357403 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199961    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199965    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199967    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199970    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199973    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199975    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199978    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199980    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199983    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199985    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199987    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199990    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199993    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199995    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.199998    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200001    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200003    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200006    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200008    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:41.358937 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200010    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200013    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200015    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200018    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200020    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200023    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200040    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200042    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200045    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200048    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200050    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200053    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200056    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200058    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200063    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200065    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200068    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200071    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200073    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:41.359573 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200075    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200078    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200080    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200083    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200086    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200089    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200092    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200571    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200577    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200580    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200583    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200586    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200589    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200591    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200594    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200597    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200599    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200602    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200605    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200608    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:41.360167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200610    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200613    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200618    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200621    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200624    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200626    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200629    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200633    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200636    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200638    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200641    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200643    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200646    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200648    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200651    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200653    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200656    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200658    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200661    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200664    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:41.360826 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200667    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200669    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200672    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200674    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200677    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200679    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200682    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200684    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200687    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200689    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200692    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200694    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200696    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200699    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200702    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200705    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200708    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200710    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200713    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:41.361568 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200716    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200719    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200721    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200724    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200726    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200729    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200732    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200734    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200739    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200742    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200745    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200747    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200750    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200753    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200756    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200758    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200761    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200764    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200766    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:41.362108 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200769    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200771    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200774    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200776    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200779    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200781    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200784    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200786    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200790    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200793    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200795    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200798    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200800    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200804    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200807    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200809    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200812    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200814    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200816    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200819    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:41.362669 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200821    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200824    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200826    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200829    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200840    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200844    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200846    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200849    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200852    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200854    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200859    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200862    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200865    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200867    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.200870    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201761    2575 flags.go:64] FLAG: --address="0.0.0.0"
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201775    2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201781    2575 flags.go:64] FLAG: --anonymous-auth="true"
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201785    2575 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201790    2575 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 03 20:38:41.363274 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201794    2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201799    2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201802    2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201806    2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201809    2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201813    2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201816    2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201820    2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201823    2575 flags.go:64] FLAG: --cgroup-root=""
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201825    2575 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201828    2575 flags.go:64] FLAG: --client-ca-file=""
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201831    2575 flags.go:64] FLAG: --cloud-provider="external"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201834    2575 flags.go:64] FLAG: --cluster-dns="[]"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201839    2575 flags.go:64] FLAG: --cluster-domain=""
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201842    2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201845    2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201848    2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201852    2575 flags.go:64] FLAG: --container-log-max-files="5"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201856    2575 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201858    2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201862    2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201865    2575 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201868    2575 flags.go:64] FLAG: --contention-profiling="false"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201871    2575 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 03 20:38:41.363812 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201874    2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201877    2575 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201880    2575 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201884    2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201887    2575 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201890    2575 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201892    2575 flags.go:64] FLAG: --enable-load-reader="false"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201895    2575 flags.go:64] FLAG: --enable-server="true"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201899    2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201904    2575 flags.go:64] FLAG: --event-burst="100"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201907    2575 flags.go:64] FLAG: --event-qps="50"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201910    2575 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201913    2575 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201917    2575 flags.go:64] FLAG: --eviction-hard=""
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201922    2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201925    2575 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201928    2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201930    2575 flags.go:64] FLAG: --eviction-soft=""
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201933    2575 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201936    2575 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201939    2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201942    2575 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201945    2575 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201948    2575 flags.go:64] FLAG: --fail-swap-on="true"
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201950    2575 flags.go:64] FLAG: --feature-gates=""
Nov 03 20:38:41.364455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201954    2575 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201957    2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201960    2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201963    2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201967    2575 flags.go:64] FLAG: --healthz-port="10248"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201970    2575 flags.go:64] FLAG: --help="false"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201973    2575 flags.go:64] FLAG: --hostname-override="ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201976    2575 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201979    2575 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201982    2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201985    2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201988    2575 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201991    2575 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201994    2575 flags.go:64] FLAG: --image-service-endpoint=""
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.201997    2575 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202000    2575 flags.go:64] FLAG: --kube-api-burst="100"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202004    2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202007    2575 flags.go:64] FLAG: --kube-api-qps="50"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202009    2575 flags.go:64] FLAG: --kube-reserved=""
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202012    2575 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202017    2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202020    2575 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202036    2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 03 20:38:41.365322 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202056    2575 flags.go:64] FLAG: --lock-file=""
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202060    2575 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202063    2575 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202066    2575 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202072    2575 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202075    2575 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202078    2575 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202080    2575 flags.go:64] FLAG: --logging-format="text"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202083    2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202087    2575 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202090    2575 flags.go:64] FLAG: --manifest-url=""
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202092    2575 flags.go:64] FLAG: --manifest-url-header=""
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202097    2575 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202100    2575 flags.go:64] FLAG: --max-open-files="1000000"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202104    2575 flags.go:64] FLAG: --max-pods="110"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202107    2575 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202110    2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202113    2575 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202115    2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202118    2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202121    2575 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202124    2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202131    2575 flags.go:64] FLAG: --node-status-max-images="50"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202134    2575 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202139    2575 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 03 20:38:41.366059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202142    2575 flags.go:64] FLAG: --pod-cidr=""
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202144    2575 flags.go:64] FLAG: --pod-manifest-path=""
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202147    2575 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202150    2575 flags.go:64] FLAG: --pods-per-core="0"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202154    2575 flags.go:64] FLAG: --port="10250"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202157    2575 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202160    2575 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-062bd8430de494aef"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202163    2575 flags.go:64] FLAG: --qos-reserved=""
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202166    2575 flags.go:64] FLAG: --read-only-port="10255"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202169    2575 flags.go:64] FLAG: --register-node="true"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202172    2575 flags.go:64] FLAG: --register-with-taints=""
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202175    2575 flags.go:64] FLAG: --registry-burst="10"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202178    2575 flags.go:64] FLAG: --registry-qps="5"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202181    2575 flags.go:64] FLAG: --reserved-cpus=""
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202184    2575 flags.go:64] FLAG: --reserved-memory=""
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202187    2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202191    2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202194    2575 flags.go:64] FLAG: --rotate-certificates="false"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202197    2575 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202199    2575 flags.go:64] FLAG: --runonce="false"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202202    2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202205    2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202208    2575 flags.go:64] FLAG: --seccomp-default="false"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202211    2575 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202214    2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202217    2575 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 03 20:38:41.366845 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202220    2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202223    2575 flags.go:64] FLAG: --storage-driver-password="root"
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202225    2575 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202228    2575 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202231    2575 flags.go:64] FLAG: --storage-driver-user="root"
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202235    2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202239    2575 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202241    2575 flags.go:64] FLAG: --system-cgroups=""
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202244    2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202250    2575 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202253    2575 flags.go:64] FLAG: --tls-cert-file=""
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202256    2575 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202260    2575 flags.go:64] FLAG: --tls-min-version=""
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202263    2575 flags.go:64] FLAG: --tls-private-key-file=""
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202266    2575 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202269    2575 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202271    2575 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202274    2575 flags.go:64] FLAG: --v="2"
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202279    2575 flags.go:64] FLAG: --version="false"
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202283    2575 flags.go:64] FLAG: --vmodule=""
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202291    2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202294    2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202425    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202428    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:41.367632 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202431    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202434    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202438    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202442    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202445    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202451    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202454    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202456    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202459    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202461    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202464    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202466    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202468    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202472    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202475    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202477    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202480    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202482    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202486    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:41.368721 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202489    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202491    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202494    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202497    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202499    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202502    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202505    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202507    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202510    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202512    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202515    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202517    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202520    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202522    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202525    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202527    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202530    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202532    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202535    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202538    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:41.369549 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202540    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:41.368984 ip-10-0-142-223 systemd[1]: Started Kubernetes Kubelet.
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202543    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202545    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202548    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202550    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202553    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202556    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202559    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202562    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202564    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202567    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202570    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202573    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202576    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202578    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202581    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202583    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202586    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202588    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:41.371087 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202591    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202593    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202595    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202598    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202601    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202603    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202605    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202608    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202610    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202613    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202615    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202618    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202620    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202623    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202626    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202628    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202631    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202633    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202636    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202639    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:41.371885 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202642    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202646    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202649    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202652    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202655    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202658    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202661    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202663    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202666    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202668    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202671    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202673    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202676    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202678    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202680    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202683    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202685    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202688    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202690    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202692    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:41.372780 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202695    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202697    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202700    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202702    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202705    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202707    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.202733    2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202767    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202772    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202775    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202778    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202780    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202785    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202788    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202790    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202793    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:41.373564 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202797    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202800    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202802    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202805    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202808    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202810    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202813    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202815    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202818    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202820    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202823    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202825    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202827    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202830    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202832    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202835    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202837    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202840    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202842    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:41.374167 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202845    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202847    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202850    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202852    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202855    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202858    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202860    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202862    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202865    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202868    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202871    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202873    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202876    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202879    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202883    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202887    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202890    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202893    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202896    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:41.375154 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202899    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202902    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202905    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202907    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202910    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202912    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202915    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202917    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202919    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202922    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202925    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202927    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202929    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202932    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202934    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202937    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202939    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202942    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202945    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202947    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:41.376468 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202949    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202952    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202956    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202958    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202962    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202966    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202970    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202972    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202975    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202977    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202980    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202982    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202985    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202988    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202990    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202993    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202996    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.202998    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203001    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:41.377451 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203003    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203006    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203008    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203011    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203013    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203016    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203018    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203021    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203035    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203038    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203040    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203043    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203046    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203049    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203051    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203055    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203058    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203060    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203063    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.203067    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:41.377897 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.210108    2575 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.210123    2575 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.210684    2575 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.210690    2575 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210770    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210775    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210779    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210783    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210787    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210789    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210792    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210795    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210798    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210800    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210803    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210806    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210808    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210811    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210813    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:41.378405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210816    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210819    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210821    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210824    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210826    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210835    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210837    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210840    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210843    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210846    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210848    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210851    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210853    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210856    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210859    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210862    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210864    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210867    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210869    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:41.378859 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210872    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210874    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210877    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210880    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210883    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210885    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210888    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210890    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210893    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210895    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210898    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210900    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210903    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210905    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210908    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210910    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210913    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210915    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210918    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:41.379405 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210927    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210930    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210934    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210938    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210941    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210944    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210947    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210954    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210957    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210960    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210963    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210966    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210968    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210971    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210973    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210976    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210978    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210981    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210984    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210986    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:41.379898 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210989    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210991    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210995    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.210999    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211001    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211004    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211007    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211009    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211012    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211015    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211017    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211020    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211022    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211049    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211053    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211056    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211058    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211061    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211063    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211067    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:41.380494 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211071    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211074    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211076    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211079    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211092    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211096    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211099    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211102    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211105    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211108    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211110    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211113    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211116    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.211147    2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211362    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211367    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:41.381396 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211370    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211373    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211375    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211378    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211381    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211383    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211388    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211391    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211394    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211397    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.211438    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212879    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212894    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212901    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212907    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212911    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212916    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212921    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212926    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:41.382127 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212931    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212936    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212940    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212945    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212955    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212959    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212963    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212968    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.212999    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213016    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213021    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213042    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213047    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213055    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213078    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213083    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213196    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213203    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213207    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:41.382902 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213212    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213216    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213220    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213224    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213228    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213232    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213237    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213242    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213246    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213250    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213254    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213258    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213263    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213267    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213271    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213275    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213279    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213283    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213287    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213291    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:41.383735 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213295    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213300    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213304    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213308    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213312    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213316    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213323    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213328    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213332    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213336    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213341    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213345    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213350    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213354    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213357    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213361    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213365    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213370    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213374    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213378    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:41.384597 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213382    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213387    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213391    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213396    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213400    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213404    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213408    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213412    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213417    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213422    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213425    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213430    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213434    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213438    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213442    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213450    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213454    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213459    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213465    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:41.385476 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213471    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213475    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213479    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213482    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213485    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213488    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213490    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.213530    2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213572    2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213578    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213581    2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213584    2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213587    2575 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213590    2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213592    2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213595    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213598    2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:38:41.386286 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213600    2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213603    2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213605    2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213608    2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213610    2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213613    2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213615    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213618    2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213621    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213624    2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213626    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213628    2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213632    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213635    2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213639    2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213643    2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213645    2575 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213648    2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213651    2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213653    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:38:41.387101 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213656    2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213659    2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213661    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213664    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213666    2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213669    2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213671    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213674    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213676    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213679    2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213681    2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213684    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213686    2575 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213689    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213691    2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213694    2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213696    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213699    2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213703    2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:38:41.387920 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213707    2575 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213710    2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213713    2575 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213716    2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213718    2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213721    2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213725    2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213729    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213733    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213736    2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213739    2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213741    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213744    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213747    2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213751    2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213754    2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213757    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213760    2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213763    2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:38:41.388639 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213765    2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213768    2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213770    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213773    2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213775    2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213778    2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213781    2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213783    2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213786    2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213788    2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213791    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213794    2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213796    2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213799    2575 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213801    2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213804    2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213807    2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213809    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213812    2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:38:41.389381 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213815    2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213819    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213823    2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213826    2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213829    2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213831    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213834    2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213838    2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213841    2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213843    2575 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213846    2575 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213848    2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213851    2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213854    2575 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213856    2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213859    2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213861    2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213864    2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213866    2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:41.213868    2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:38:41.390102 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.213929    2575 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 03 20:38:41.390857 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.290110    2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 03 20:38:41.390857 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.293046    2575 server.go:1014] "Starting client certificate rotation"
Nov 03 20:38:41.390857 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.293152    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 20:38:41.390857 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.293192    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 20:38:41.390857 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.295147    2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 20:38:41.390857 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.296828    2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 20:38:41.390857 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.308486    2575 log.go:25] "Validated CRI v1 runtime API"
Nov 03 20:38:41.390857 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.314814    2575 log.go:25] "Validated CRI v1 image API"
Nov 03 20:38:41.390857 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.315882    2575 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 03 20:38:41.390857 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.323926    2575 fs.go:135] Filesystem UUIDs: map[4e648b96-e44a-4406-8298-7e74fbc6b36c:/dev/nvme0n1p4 5ddd11bb-eac6-4f1b-8221-a69a3211d16f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Nov 03 20:38:41.390857 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.323947    2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.330367    2575 manager.go:217] Machine: {Timestamp:2026-11-03 20:38:41.328859794 +0000 UTC m=+0.402369923 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099271 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e608996836c052a1ee82039a5c577 SystemUUID:ec2e6089-9683-6c05-2a1e-e82039a5c577 BootID:26f56577-9871-47c0-863e-c4eaf49187ee Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:15:0d:20:47:25 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:15:0d:20:47:25 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:86:95:28:d3:b6:a4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.330904    2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.330976    2575 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.332856    2575 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.332878    2575 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-223.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.333022    2575 topology_manager.go:143] "Creating topology manager with none policy"
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.333047    2575 container_manager_linux.go:311] "Creating device plugin manager"
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.333060    2575 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.333636    2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.334369    2575 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.337873    2575 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.337993    2575 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.338659    2575 kubelet.go:493] "Attempting to sync node with API server"
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.338669    2575 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.338682    2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.338691    2575 kubelet.go:405] "Adding apiserver pod source"
Nov 03 20:38:41.391416 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.338705    2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.339128    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.342313    2575 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.347119    2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.347914    2575 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.347943    2575 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.348125    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.348137    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.348142    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.348148    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.348153    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.348159    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.348729    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.348736    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.348743    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.348749    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.348816    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.348824    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.349666    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.349675    2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.356764    2575 server.go:1263] "Started kubelet"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.356844    2575 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.361079    2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.361384    2575 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.362797    2575 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.363946    2575 server.go:317] "Adding debug handlers to kubelet server"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.366861    2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.367345    2575 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.367443    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.368219    2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.369669    2575 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.369688    2575 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.369823    2575 reconstruct.go:97] "Volume reconstruction finished"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.369831    2575 reconciler.go:29] "Reconciler: start to sync state"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.372229    2575 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.372449    2575 factory.go:55] Registering systemd factory
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.372460    2575 factory.go:223] Registration of the systemd container factory successfully
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.372928    2575 factory.go:153] Registering CRI-O factory
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.372944    2575 factory.go:223] Registration of the crio container factory successfully
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.372998    2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.373044    2575 factory.go:103] Registering Raw factory
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.373058    2575 manager.go:1199] Started watching for new ooms in manager
Nov 03 20:38:41.391892 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.374173    2575 manager.go:319] Starting recovery of all containers
Nov 03 20:38:41.393153 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.392048    2575 manager.go:324] Recovery completed
Nov 03 20:38:41.396785 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.396769    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:38:41.398317 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.398289    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:38:41.398387 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.398331    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:38:41.398387 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.398344    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:38:41.398842 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.398829    2575 cpu_manager.go:226] "Starting" policy="none"
Nov 03 20:38:41.398881 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.398842    2575 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 03 20:38:41.398881 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.398858    2575 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 03 20:38:41.398932 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.398885    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:38:41.399523 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.399508    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:38:41.399582 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.399533    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:38:41.399582 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.399542    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:38:41.401131 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.401119    2575 policy_none.go:50] "Start"
Nov 03 20:38:41.401180 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.401134    2575 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 03 20:38:41.401180 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.401144    2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 03 20:38:41.403128 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.403118    2575 policy_none.go:44] "Start"
Nov 03 20:38:41.445716 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.445566    2575 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 03 20:38:41.450368 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.450355    2575 manager.go:341] "Starting Device Plugin manager"
Nov 03 20:38:41.450459 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.450391    2575 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 03 20:38:41.450459 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.450404    2575 server.go:87] "Starting device plugin registration server"
Nov 03 20:38:41.450633 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.450621    2575 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 03 20:38:41.450675 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.450633    2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 03 20:38:41.450763 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.450747    2575 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 03 20:38:41.450881 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.450862    2575 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 03 20:38:41.450881 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.450876    2575 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 03 20:38:41.451501 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.451470    2575 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 03 20:38:41.451586 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.451527    2575 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:38:41.499118 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.499093    2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-223.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 03 20:38:41.501055 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.499093    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-223.us-west-1.compute.internal.18e4a37fe87930cb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-223.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-223.us-west-1.compute.internal,},FirstTimestamp:2026-11-03 20:38:41.356730571 +0000 UTC m=+0.430240699,LastTimestamp:2026-11-03 20:38:41.356730571 +0000 UTC m=+0.430240699,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-223.us-west-1.compute.internal,}"
Nov 03 20:38:41.509909 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.509886    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 03 20:38:41.511151 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.511138    2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 03 20:38:41.511207 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.511158    2575 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 03 20:38:41.511207 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.511174    2575 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 03 20:38:41.511276 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.511206    2575 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 03 20:38:41.551021 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.550971    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:38:41.551767 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.551749    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:38:41.551818 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.551783    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:38:41.551818 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.551794    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:38:41.551818 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.551816    2575 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.557928 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.557906    2575 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 03 20:38:41.557928 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.557905    2575 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-223.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 03 20:38:41.557928 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.557910    2575 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 03 20:38:41.558120 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.557970    2575 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-223.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 03 20:38:41.558120 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.557922    2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-223.us-west-1.compute.internal.18e4a37feaf3b9bd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-223.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-223.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-223.us-west-1.compute.internal,},FirstTimestamp:2026-11-03 20:38:41.398315453 +0000 UTC m=+0.471825581,LastTimestamp:2026-11-03 20:38:41.398315453 +0000 UTC m=+0.471825581,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-223.us-west-1.compute.internal,}"
Nov 03 20:38:41.575469 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.575450    2575 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 03 20:38:41.575514 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.575499    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4d5g2"
Nov 03 20:38:41.611488 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.611463    2575 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal"]
Nov 03 20:38:41.611574 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.611518    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:38:41.612280 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.612263    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:38:41.612354 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.612294    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:38:41.612354 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.612307    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:38:41.614510 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.614497    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:38:41.614666 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.614653    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.614707 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.614682    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:38:41.615164 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.615148    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:38:41.615274 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.615178    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:38:41.615274 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.615195    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:38:41.615274 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.615205    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:38:41.615274 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.615236    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:38:41.615274 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.615247    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:38:41.617259 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.617245    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.617320 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.617269    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:38:41.617907 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.617883    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:38:41.617973 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.617911    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:38:41.617973 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.617926    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:38:41.639244 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.639228    2575 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found" node="ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.641211 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.641196    2575 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found" node="ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.659398 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.659380    2575 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:38:41.665858 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.665844    2575 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.665931 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.665862    2575 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-223.us-west-1.compute.internal\": node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:38:41.672739 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.672723    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61bc8eba7840edd90e06bc583d329627-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"61bc8eba7840edd90e06bc583d329627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.672793 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.672747    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61bc8eba7840edd90e06bc583d329627-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"61bc8eba7840edd90e06bc583d329627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.672793 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.672767    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/88d138da692bc0e2c453d9d2e3886bfc-config\") pod \"kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"88d138da692bc0e2c453d9d2e3886bfc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.773002 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.772979    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61bc8eba7840edd90e06bc583d329627-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"61bc8eba7840edd90e06bc583d329627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.773111 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.773005    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61bc8eba7840edd90e06bc583d329627-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"61bc8eba7840edd90e06bc583d329627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.773111 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.773023    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/88d138da692bc0e2c453d9d2e3886bfc-config\") pod \"kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"88d138da692bc0e2c453d9d2e3886bfc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.773111 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.773071    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61bc8eba7840edd90e06bc583d329627-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"61bc8eba7840edd90e06bc583d329627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.773111 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.773081    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61bc8eba7840edd90e06bc583d329627-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"61bc8eba7840edd90e06bc583d329627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.773111 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.773071    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/88d138da692bc0e2c453d9d2e3886bfc-config\") pod \"kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"88d138da692bc0e2c453d9d2e3886bfc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.776839 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.776815    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:38:41.877721 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.877673    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:38:41.942647 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.942626    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.944786 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:41.944769    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:41.978110 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:41.978091    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:38:42.079021 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:42.078998    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:38:42.179867 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:42.179815    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:38:42.280798 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:42.280773    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:38:42.293139 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:42.292907    2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 03 20:38:42.296373 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:42.293181    2575 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a0165055bbafc464b85f5db0d8234891-51f743d41745d646.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-142-223.us-west-1.compute.internal.18e4a37feaf40b0a\": read tcp 10.0.142.223:33776->3.21.224.121:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-142-223.us-west-1.compute.internal.18e4a37feaf40b0a  default   4991 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-223.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-223.us-west-1.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-223.us-west-1.compute.internal,},FirstTimestamp:2026-11-03 20:38:41 +0000 UTC,LastTimestamp:2026-11-03 20:38:41.612301251 +0000 UTC m=+0.685811378,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-223.us-west-1.compute.internal,}"
Nov 03 20:38:42.296373 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:42.294435    2575 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 03 20:38:42.381014 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:42.380989    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:38:42.481620 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:42.481570    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:38:42.579526 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:42.579495    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 20:33:42 +0000 UTC" deadline="2028-07-15 06:51:53.19239019 +0000 UTC"
Nov 03 20:38:42.579526 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:42.579525    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14866h13m10.612870593s"
Nov 03 20:38:42.582622 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:42.582607    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:38:42.604273 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:42.604255    2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:38:42.605046 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:42.605020    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:38:42.605099 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:42.605063    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:38:42.605099 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:42.605079    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:38:42.683066 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:42.683038    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:38:42.757642 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:42.757574    2575 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:38:42.783149 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:42.783127    2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:38:42.839493 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:42.839473    2575 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:38:42.867796 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:42.867778    2575 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:42.926568 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:42.926542    2575 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 20:38:42.927532 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:42.927514    2575 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:38:42.986886 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:42.986861    2575 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 20:38:43.023769 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.023720    2575 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:38:43.039204 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:43.039175    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88d138da692bc0e2c453d9d2e3886bfc.slice/crio-04aa687fe51e5801032b5505b2ed2d459b20773723b08d7fefc0815ce364e082 WatchSource:0}: Error finding container 04aa687fe51e5801032b5505b2ed2d459b20773723b08d7fefc0815ce364e082: Status 404 returned error can't find the container with id 04aa687fe51e5801032b5505b2ed2d459b20773723b08d7fefc0815ce364e082
Nov 03 20:38:43.039789 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:43.039767    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61bc8eba7840edd90e06bc583d329627.slice/crio-e88f6a237b09b306079b4a9f5aa3eeca9a6fe5b518e5c10d63ddb151a39c9d3d WatchSource:0}: Error finding container e88f6a237b09b306079b4a9f5aa3eeca9a6fe5b518e5c10d63ddb151a39c9d3d: Status 404 returned error can't find the container with id e88f6a237b09b306079b4a9f5aa3eeca9a6fe5b518e5c10d63ddb151a39c9d3d
Nov 03 20:38:43.044850 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.044827    2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 20:38:43.339627 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.339561    2575 apiserver.go:52] "Watching apiserver"
Nov 03 20:38:43.363324 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.363298    2575 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:38:43.368417 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.368405    2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 03 20:38:43.442559 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.442536    2575 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 03 20:38:43.442837 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.442820    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-585md","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h","openshift-dns/node-resolver-pbhv8","kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal","openshift-multus/multus-additional-cni-plugins-lxzlf","openshift-multus/network-metrics-daemon-hmnm7","openshift-ovn-kubernetes/ovnkube-node-pwvcf","kube-system/global-pull-secret-syncer-8jhcx","openshift-image-registry/node-ca-9hqmv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal","openshift-network-diagnostics/network-check-target-l9p4t","openshift-network-operator/iptables-alerter-4dwkb","kube-system/konnectivity-agent-h952v","openshift-cluster-node-tuning-operator/tuned-dm2zp"]
Nov 03 20:38:43.446660 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.446641    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:43.446742 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:43.446724    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:38:43.448773 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.448756    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:38:43.450978 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.450962    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:38:43.453283 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.453268    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.455733 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.455535    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:38:43.457847 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.457829    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-585md"
Nov 03 20:38:43.460088 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.460071    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.462524 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.462505    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.464630 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.464617    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:43.464701 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:43.464671    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:38:43.467189 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.467169    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.469461 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.469443    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:38:43.469535 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:43.469503    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:38:43.471540 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.471526    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:38:43.481688 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.481665    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-k8s-cni-cncf-io\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.481767 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.481700    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-conf-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.481767 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.481722    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-systemd\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.481767 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.481756    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-var-lib-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.481883 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.481843    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:43.481930 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.481879    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-system-cni-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.481930 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.481904    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-cni-bin\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.482047 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.481954    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-multus-certs\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.482047 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.481988    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-kubelet\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.482047 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482014    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-cni-bin\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.482208 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482056    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl9lr\" (UniqueName: \"kubernetes.io/projected/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-kube-api-access-sl9lr\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.482208 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482081    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-registration-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.482208 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482109    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-tmp-dir\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:38:43.482208 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482135    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlfjt\" (UniqueName: \"kubernetes.io/projected/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-kube-api-access-qlfjt\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:38:43.482208 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482163    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10a65b13-a9e4-4355-8bac-fab245388f16-cni-binary-copy\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.482208 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482186    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-etc-kubernetes\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.482480 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482209    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.482480 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482233    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.482480 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482258    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovnkube-script-lib\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.482480 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482282    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-cni-multus\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.482480 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482304    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10a65b13-a9e4-4355-8bac-fab245388f16-multus-daemon-config\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.482480 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482326    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-slash\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.482480 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482350    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:43.482480 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482373    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-kubernetes\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.482480 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482395    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-systemd\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.482480 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482425    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-hosts-file\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:38:43.482480 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482447    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b6fe25b0-03dc-434d-957b-5f25c8d5c141-konnectivity-ca\") pod \"konnectivity-agent-h952v\" (UID: \"b6fe25b0-03dc-434d-957b-5f25c8d5c141\") " pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:38:43.482480 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482464    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-systemd-units\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.482480 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482477    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-node-log\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482492    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-cni-netd\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482528    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-socket-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482556    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-kubelet\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482581    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482614    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-env-overrides\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482645    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovn-node-metrics-cert\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482662    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysctl-d\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482705    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-lib-modules\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482741    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-sys-fs\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482782    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbpzw\" (UniqueName: \"kubernetes.io/projected/10a65b13-a9e4-4355-8bac-fab245388f16-kube-api-access-kbpzw\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482810    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-system-cni-dir\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482847    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5vgd\" (UniqueName: \"kubernetes.io/projected/0ffcf342-bccb-4362-b79e-8faf3d337b17-kube-api-access-t5vgd\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482864    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87s22\" (UniqueName: \"kubernetes.io/projected/57913479-90b9-4db4-90f1-81cb81c2e5bb-kube-api-access-87s22\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482881    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbw7c\" (UniqueName: \"kubernetes.io/projected/e658bf30-634f-4db1-9b41-280a746540a8-kube-api-access-dbw7c\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482917    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-cnibin\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.482984 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482956    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.482987    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b6fe25b0-03dc-434d-957b-5f25c8d5c141-agent-certs\") pod \"konnectivity-agent-h952v\" (UID: \"b6fe25b0-03dc-434d-957b-5f25c8d5c141\") " pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483014    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysconfig\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483064    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-cni-binary-copy\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483079    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9353a680-00bf-4d3a-8437-5a959a40a682-kubelet-config\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483094    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-tmp\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483107    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e658bf30-634f-4db1-9b41-280a746540a8-serviceca\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483120    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-modprobe-d\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483141    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-cnibin\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483158    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-ovn\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483170    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-log-socket\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483183    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-sys\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483196    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-var-lib-kubelet\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483234    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-host\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483259    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-device-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483275    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-run-netns\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483323    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-etc-selinux\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.483780 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483340    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-socket-dir-parent\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483355    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9353a680-00bf-4d3a-8437-5a959a40a682-dbus\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483375    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lhxc\" (UniqueName: \"kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc\") pod \"network-check-target-l9p4t\" (UID: \"6b982774-43c6-45ad-a3e3-77b7bc3d534e\") " pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483389    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmdh2\" (UniqueName: \"kubernetes.io/projected/32cf1fce-f61e-4870-aff9-a9171e90b09f-kube-api-access-wmdh2\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483404    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-netns\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483418    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-hostroot\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483442    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483458    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-etc-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483472    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bffp\" (UniqueName: \"kubernetes.io/projected/11516670-b4cd-43cc-8c41-64e5bfed2b5a-kube-api-access-9bffp\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483487    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysctl-conf\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483500    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-run\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483517    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e658bf30-634f-4db1-9b41-280a746540a8-host\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483555    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-cni-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483572    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483603    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovnkube-config\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483635    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-tuned\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.484648 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.483656    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.501131 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.501112    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 03 20:38:43.501210 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.501184    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 03 20:38:43.501246 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.501209    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 03 20:38:43.501851 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.501836    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-czwwq\""
Nov 03 20:38:43.503497 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.503474    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 03 20:38:43.503564 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.503518    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 03 20:38:43.503564 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.503535    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-29dtl\""
Nov 03 20:38:43.504763 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.504739    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 03 20:38:43.504974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.504959    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zxrmv\""
Nov 03 20:38:43.514849 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.514808    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal" event={"ID":"61bc8eba7840edd90e06bc583d329627","Type":"ContainerStarted","Data":"e88f6a237b09b306079b4a9f5aa3eeca9a6fe5b518e5c10d63ddb151a39c9d3d"}
Nov 03 20:38:43.515829 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.515812    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal" event={"ID":"88d138da692bc0e2c453d9d2e3886bfc","Type":"ContainerStarted","Data":"04aa687fe51e5801032b5505b2ed2d459b20773723b08d7fefc0815ce364e082"}
Nov 03 20:38:43.569403 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.569388    2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 03 20:38:43.584145 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584118    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-modprobe-d\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.584250 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584148    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-cnibin\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.584250 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584165    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-ovn\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.584250 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584180    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-log-socket\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.584250 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584209    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-modprobe-d\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.584250 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584225    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-sys\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.584250 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584214    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-log-socket\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.584250 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584235    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-ovn\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.584250 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584240    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-cnibin\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584255    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-var-lib-kubelet\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584281    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-host\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584315    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-sys\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584307    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-device-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584339    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-var-lib-kubelet\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584352    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-run-netns\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584387    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-host\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584391    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-device-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584425    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-run-netns\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584428    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-etc-selinux\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584474    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j726r\" (UniqueName: \"kubernetes.io/projected/e5e0251d-3ac4-4b42-9175-39ffd668896f-kube-api-access-j726r\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584489    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-etc-selinux\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584501    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-socket-dir-parent\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584528    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9353a680-00bf-4d3a-8437-5a959a40a682-dbus\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584556    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhxc\" (UniqueName: \"kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc\") pod \"network-check-target-l9p4t\" (UID: \"6b982774-43c6-45ad-a3e3-77b7bc3d534e\") " pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584569    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-socket-dir-parent\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.584626 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584582    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmdh2\" (UniqueName: \"kubernetes.io/projected/32cf1fce-f61e-4870-aff9-a9171e90b09f-kube-api-access-wmdh2\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584605    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9353a680-00bf-4d3a-8437-5a959a40a682-dbus\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584606    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-netns\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584626    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-hostroot\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584633    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-netns\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584644    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584668    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-etc-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584684    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bffp\" (UniqueName: \"kubernetes.io/projected/11516670-b4cd-43cc-8c41-64e5bfed2b5a-kube-api-access-9bffp\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584698    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysctl-conf\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584681    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-hostroot\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584717    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-etc-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584756    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-run\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584778    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e658bf30-634f-4db1-9b41-280a746540a8-host\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584798    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-cni-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584808    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-run\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584830    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e658bf30-634f-4db1-9b41-280a746540a8-host\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584842    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584876    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.585334 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584880    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovnkube-config\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584853    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysctl-conf\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584905    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-cni-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584912    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-tuned\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584947    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.584965    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5e0251d-3ac4-4b42-9175-39ffd668896f-iptables-alerter-script\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585000    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-k8s-cni-cncf-io\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585014    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-conf-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585047    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-systemd\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585082    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-k8s-cni-cncf-io\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585084    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-var-lib-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585095    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585108    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-var-lib-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585111    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585131    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-conf-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585169    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-systemd\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585177    2575 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585192    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-system-cni-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.585974 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:43.585203    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585213    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-cni-bin\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585228    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-multus-certs\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585233    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-system-cni-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585246    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-multus-certs\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:43.585260    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs podName:57913479-90b9-4db4-90f1-81cb81c2e5bb nodeName:}" failed. No retries permitted until 2026-11-03 20:38:44.085230231 +0000 UTC m=+3.158740349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs") pod "network-metrics-daemon-hmnm7" (UID: "57913479-90b9-4db4-90f1-81cb81c2e5bb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585265    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-cni-bin\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585293    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-kubelet\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585317    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-cni-bin\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585342    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl9lr\" (UniqueName: \"kubernetes.io/projected/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-kube-api-access-sl9lr\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585347    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-kubelet\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585364    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-cni-bin\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585367    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-registration-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585392    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-tmp-dir\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585401    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-registration-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585407    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlfjt\" (UniqueName: \"kubernetes.io/projected/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-kube-api-access-qlfjt\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585435    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10a65b13-a9e4-4355-8bac-fab245388f16-cni-binary-copy\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.586757 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585461    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-etc-kubernetes\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585483    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585511    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585522    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-etc-kubernetes\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585574    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585601    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovnkube-script-lib\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585628    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-cni-multus\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585639    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-tmp-dir\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585643    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585654    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10a65b13-a9e4-4355-8bac-fab245388f16-multus-daemon-config\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585680    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-slash\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585684    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-cni-multus\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585705    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585712    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-slash\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585729    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-kubernetes\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585748    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-systemd\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585793    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-systemd\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585804    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-kubernetes\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.587331 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:43.585810    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585834    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-hosts-file\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:43.585866    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret podName:9353a680-00bf-4d3a-8437-5a959a40a682 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:44.085851451 +0000 UTC m=+3.159361569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret") pod "global-pull-secret-syncer-8jhcx" (UID: "9353a680-00bf-4d3a-8437-5a959a40a682") : object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585873    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-hosts-file\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585881    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b6fe25b0-03dc-434d-957b-5f25c8d5c141-konnectivity-ca\") pod \"konnectivity-agent-h952v\" (UID: \"b6fe25b0-03dc-434d-957b-5f25c8d5c141\") " pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585897    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-systemd-units\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585912    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-node-log\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585933    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-cni-netd\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585957    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-socket-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585961    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-systemd-units\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.585980    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-kubelet\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586003    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-node-log\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586006    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5e0251d-3ac4-4b42-9175-39ffd668896f-host-slash\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586044    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-cni-netd\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586046    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-kubelet\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586069    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586085    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-socket-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.587889 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586093    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-env-overrides\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586112    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovn-node-metrics-cert\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586126    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysctl-d\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586141    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-lib-modules\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586155    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-sys-fs\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586208    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbpzw\" (UniqueName: \"kubernetes.io/projected/10a65b13-a9e4-4355-8bac-fab245388f16-kube-api-access-kbpzw\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586224    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-system-cni-dir\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586239    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5vgd\" (UniqueName: \"kubernetes.io/projected/0ffcf342-bccb-4362-b79e-8faf3d337b17-kube-api-access-t5vgd\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586243    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysctl-d\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586261    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87s22\" (UniqueName: \"kubernetes.io/projected/57913479-90b9-4db4-90f1-81cb81c2e5bb-kube-api-access-87s22\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586281    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-lib-modules\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586293    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-system-cni-dir\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586286    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbw7c\" (UniqueName: \"kubernetes.io/projected/e658bf30-634f-4db1-9b41-280a746540a8-kube-api-access-dbw7c\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586314    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-sys-fs\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586328    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b6fe25b0-03dc-434d-957b-5f25c8d5c141-konnectivity-ca\") pod \"konnectivity-agent-h952v\" (UID: \"b6fe25b0-03dc-434d-957b-5f25c8d5c141\") " pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586337    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-cnibin\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586367    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.588428 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586391    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b6fe25b0-03dc-434d-957b-5f25c8d5c141-agent-certs\") pod \"konnectivity-agent-h952v\" (UID: \"b6fe25b0-03dc-434d-957b-5f25c8d5c141\") " pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:38:43.588954 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586399    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-cnibin\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.588954 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586423    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysconfig\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.588954 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586450    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-cni-binary-copy\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.588954 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586472    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9353a680-00bf-4d3a-8437-5a959a40a682-kubelet-config\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:43.588954 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586495    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-tmp\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.588954 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586507    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.588954 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586518    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e658bf30-634f-4db1-9b41-280a746540a8-serviceca\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:38:43.588954 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586510    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysconfig\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.588954 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.586555    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9353a680-00bf-4d3a-8437-5a959a40a682-kubelet-config\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:43.588954 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.587116    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e658bf30-634f-4db1-9b41-280a746540a8-serviceca\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:38:43.588954 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.588733    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-tuned\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.588954 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.588763    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-tmp\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.589348 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.588965    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b6fe25b0-03dc-434d-957b-5f25c8d5c141-agent-certs\") pod \"konnectivity-agent-h952v\" (UID: \"b6fe25b0-03dc-434d-957b-5f25c8d5c141\") " pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:38:43.595012 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.594969    2575 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 03 20:38:43.595012 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.595002    2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hkw4h"
Nov 03 20:38:43.601754 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.601734    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 03 20:38:43.601861 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.601804    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:38:43.601861 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.601832    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 03 20:38:43.601969 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.601869    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 03 20:38:43.601969 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.601883    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 03 20:38:43.601969 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.601871    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 03 20:38:43.601969 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.601935    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 03 20:38:43.601969 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.601949    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 03 20:38:43.602231 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602064    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 03 20:38:43.602350 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602319    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 03 20:38:43.602414 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602332    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 03 20:38:43.602472 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602457    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:38:43.602601 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602586    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 03 20:38:43.602601 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602598    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r6k78\""
Nov 03 20:38:43.602710 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602628    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dm4wl\""
Nov 03 20:38:43.602710 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602654    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 03 20:38:43.602804 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602723    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 03 20:38:43.602804 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602724    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4xfms\""
Nov 03 20:38:43.602804 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602748    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 03 20:38:43.602804 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602746    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-kqp9s\""
Nov 03 20:38:43.602804 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602755    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 03 20:38:43.602804 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602778    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 03 20:38:43.602972 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602836    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nnzhp\""
Nov 03 20:38:43.602972 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602910    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 03 20:38:43.602972 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602936    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 03 20:38:43.603071 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.602976    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8nf42\""
Nov 03 20:38:43.603071 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.603042    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 03 20:38:43.605598 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.605579    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.605641 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.605586    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovnkube-config\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.605904 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.605890    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10a65b13-a9e4-4355-8bac-fab245388f16-cni-binary-copy\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.606377 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.606356    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10a65b13-a9e4-4355-8bac-fab245388f16-multus-daemon-config\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.606704 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.606666    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-env-overrides\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.606787 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.606752    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovnkube-script-lib\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.606968 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.606950    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-cni-binary-copy\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.607056 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.606996    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.608049 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.608016    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovn-node-metrics-cert\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.645149 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:43.645122    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 20:38:43.645149 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:43.645148    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 20:38:43.645299 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:43.645159    2575 projected.go:196] Error preparing data for projected volume kube-api-access-5lhxc for pod openshift-network-diagnostics/network-check-target-l9p4t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:43.645299 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:43.645233    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc podName:6b982774-43c6-45ad-a3e3-77b7bc3d534e nodeName:}" failed. No retries permitted until 2026-11-03 20:38:44.145216669 +0000 UTC m=+3.218726803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5lhxc" (UniqueName: "kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc") pod "network-check-target-l9p4t" (UID: "6b982774-43c6-45ad-a3e3-77b7bc3d534e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:43.650895 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.648701    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbpzw\" (UniqueName: \"kubernetes.io/projected/10a65b13-a9e4-4355-8bac-fab245388f16-kube-api-access-kbpzw\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:38:43.650895 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.648736    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmdh2\" (UniqueName: \"kubernetes.io/projected/32cf1fce-f61e-4870-aff9-a9171e90b09f-kube-api-access-wmdh2\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:43.650895 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.648766    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlfjt\" (UniqueName: \"kubernetes.io/projected/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-kube-api-access-qlfjt\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:38:43.650895 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.648875    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bffp\" (UniqueName: \"kubernetes.io/projected/11516670-b4cd-43cc-8c41-64e5bfed2b5a-kube-api-access-9bffp\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:43.650895 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.649050    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbw7c\" (UniqueName: \"kubernetes.io/projected/e658bf30-634f-4db1-9b41-280a746540a8-kube-api-access-dbw7c\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:38:43.650895 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.649077    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87s22\" (UniqueName: \"kubernetes.io/projected/57913479-90b9-4db4-90f1-81cb81c2e5bb-kube-api-access-87s22\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:43.650895 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.649616    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl9lr\" (UniqueName: \"kubernetes.io/projected/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-kube-api-access-sl9lr\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:43.650895 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.649632    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5vgd\" (UniqueName: \"kubernetes.io/projected/0ffcf342-bccb-4362-b79e-8faf3d337b17-kube-api-access-t5vgd\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:43.687551 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.687524    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j726r\" (UniqueName: \"kubernetes.io/projected/e5e0251d-3ac4-4b42-9175-39ffd668896f-kube-api-access-j726r\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:38:43.687642 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.687575    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5e0251d-3ac4-4b42-9175-39ffd668896f-iptables-alerter-script\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:38:43.687642 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.687635    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5e0251d-3ac4-4b42-9175-39ffd668896f-host-slash\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:38:43.687739 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.687700    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5e0251d-3ac4-4b42-9175-39ffd668896f-host-slash\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:38:43.688066 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:43.688045    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5e0251d-3ac4-4b42-9175-39ffd668896f-iptables-alerter-script\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:38:45.088842 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.088821    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j726r\" (UniqueName: \"kubernetes.io/projected/e5e0251d-3ac4-4b42-9175-39ffd668896f-kube-api-access-j726r\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:38:45.099761 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.099748    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:38:45.105679 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.105662    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:38:45.114079 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:45.114060    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6fe25b0_03dc_434d_957b_5f25c8d5c141.slice/crio-a24e27a43277303c542a3fab007c981392e4c6cb4eca6bb9dbef0a265fbdc3c5 WatchSource:0}: Error finding container a24e27a43277303c542a3fab007c981392e4c6cb4eca6bb9dbef0a265fbdc3c5: Status 404 returned error can't find the container with id a24e27a43277303c542a3fab007c981392e4c6cb4eca6bb9dbef0a265fbdc3c5
Nov 03 20:38:45.115754 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.115736    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:38:45.117036 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.117018    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:38:45.123809 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.123785    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-585md"
Nov 03 20:38:45.126117 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:45.126084    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32cf1fce_f61e_4870_aff9_a9171e90b09f.slice/crio-62995e99c79deae78ab49a3f98e520833e6ff056e23073fe3b09ddfc5f8a8e5d WatchSource:0}: Error finding container 62995e99c79deae78ab49a3f98e520833e6ff056e23073fe3b09ddfc5f8a8e5d: Status 404 returned error can't find the container with id 62995e99c79deae78ab49a3f98e520833e6ff056e23073fe3b09ddfc5f8a8e5d
Nov 03 20:38:45.128904 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:45.128820    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853d1e90_3b6e_4bb5_b6c5_82b1ce141e43.slice/crio-d27d19c6586f35ebee131ffe2dc27cd671d13753bdf7c0c7e950c8f166acbf38 WatchSource:0}: Error finding container d27d19c6586f35ebee131ffe2dc27cd671d13753bdf7c0c7e950c8f166acbf38: Status 404 returned error can't find the container with id d27d19c6586f35ebee131ffe2dc27cd671d13753bdf7c0c7e950c8f166acbf38
Nov 03 20:38:45.132813 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:45.132797    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a65b13_a9e4_4355_8bac_fab245388f16.slice/crio-e92ecfcfeb535a68e348a0311cf7def199e9eb25188161b246dfbfc28236c9d5 WatchSource:0}: Error finding container e92ecfcfeb535a68e348a0311cf7def199e9eb25188161b246dfbfc28236c9d5: Status 404 returned error can't find the container with id e92ecfcfeb535a68e348a0311cf7def199e9eb25188161b246dfbfc28236c9d5
Nov 03 20:38:45.146689 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.146674    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:38:45.149810 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.149797    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:38:45.152148 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.152128    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:38:45.155827 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:45.155810    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1169f5bf_4ec8_4f85_aba9_857c3a666a3f.slice/crio-b00a6ae18d7a6685385b25e576aea26607a64f174d85e09d728358d0715981f1 WatchSource:0}: Error finding container b00a6ae18d7a6685385b25e576aea26607a64f174d85e09d728358d0715981f1: Status 404 returned error can't find the container with id b00a6ae18d7a6685385b25e576aea26607a64f174d85e09d728358d0715981f1
Nov 03 20:38:45.159237 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:45.159216    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ffcf342_bccb_4362_b79e_8faf3d337b17.slice/crio-d721f687bcff21f43e7a90eb4e2129c0f46ee36289c481e438e4e3cabbdc9ff3 WatchSource:0}: Error finding container d721f687bcff21f43e7a90eb4e2129c0f46ee36289c481e438e4e3cabbdc9ff3: Status 404 returned error can't find the container with id d721f687bcff21f43e7a90eb4e2129c0f46ee36289c481e438e4e3cabbdc9ff3
Nov 03 20:38:45.161170 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:45.161104    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e0251d_3ac4_4b42_9175_39ffd668896f.slice/crio-264d7531f8500f215d53e74449911b84b203f7bbeb4549e3f9ab721c1d5ce85f WatchSource:0}: Error finding container 264d7531f8500f215d53e74449911b84b203f7bbeb4549e3f9ab721c1d5ce85f: Status 404 returned error can't find the container with id 264d7531f8500f215d53e74449911b84b203f7bbeb4549e3f9ab721c1d5ce85f
Nov 03 20:38:45.162270 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.162168    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:38:45.170725 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:38:45.170707    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11516670_b4cd_43cc_8c41_64e5bfed2b5a.slice/crio-579eab1e7331b2200a1eb348da970ca5a5300f140c69684d3c64ce632b879e61 WatchSource:0}: Error finding container 579eab1e7331b2200a1eb348da970ca5a5300f140c69684d3c64ce632b879e61: Status 404 returned error can't find the container with id 579eab1e7331b2200a1eb348da970ca5a5300f140c69684d3c64ce632b879e61
Nov 03 20:38:45.429455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.429381    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:45.429455 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.429427    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:45.429647 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:45.429533    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:45.429647 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:45.429574    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret podName:9353a680-00bf-4d3a-8437-5a959a40a682 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:46.429563146 +0000 UTC m=+4.164473005 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret") pod "global-pull-secret-syncer-8jhcx" (UID: "9353a680-00bf-4d3a-8437-5a959a40a682") : object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:45.429647 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:45.429643    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:45.429813 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:45.429675    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs podName:57913479-90b9-4db4-90f1-81cb81c2e5bb nodeName:}" failed. No retries permitted until 2026-11-03 20:38:46.429663126 +0000 UTC m=+4.164572990 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs") pod "network-metrics-daemon-hmnm7" (UID: "57913479-90b9-4db4-90f1-81cb81c2e5bb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:45.530523 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.530490    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhxc\" (UniqueName: \"kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc\") pod \"network-check-target-l9p4t\" (UID: \"6b982774-43c6-45ad-a3e3-77b7bc3d534e\") " pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:38:45.530704 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:45.530619    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 20:38:45.530704 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:45.530655    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 20:38:45.530704 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:45.530671    2575 projected.go:196] Error preparing data for projected volume kube-api-access-5lhxc for pod openshift-network-diagnostics/network-check-target-l9p4t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:45.530864 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:45.530733    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc podName:6b982774-43c6-45ad-a3e3-77b7bc3d534e nodeName:}" failed. No retries permitted until 2026-11-03 20:38:46.530713361 +0000 UTC m=+4.265623216 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5lhxc" (UniqueName: "kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc") pod "network-check-target-l9p4t" (UID: "6b982774-43c6-45ad-a3e3-77b7bc3d534e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:45.857976 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.857888    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf" event={"ID":"11516670-b4cd-43cc-8c41-64e5bfed2b5a","Type":"ContainerStarted","Data":"579eab1e7331b2200a1eb348da970ca5a5300f140c69684d3c64ce632b879e61"}
Nov 03 20:38:45.859332 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.859298    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxzlf" event={"ID":"0ffcf342-bccb-4362-b79e-8faf3d337b17","Type":"ContainerStarted","Data":"d721f687bcff21f43e7a90eb4e2129c0f46ee36289c481e438e4e3cabbdc9ff3"}
Nov 03 20:38:45.860695 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.860664    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dm2zp" event={"ID":"1169f5bf-4ec8-4f85-aba9-857c3a666a3f","Type":"ContainerStarted","Data":"b00a6ae18d7a6685385b25e576aea26607a64f174d85e09d728358d0715981f1"}
Nov 03 20:38:45.862017 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.861993    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-585md" event={"ID":"10a65b13-a9e4-4355-8bac-fab245388f16","Type":"ContainerStarted","Data":"e92ecfcfeb535a68e348a0311cf7def199e9eb25188161b246dfbfc28236c9d5"}
Nov 03 20:38:45.863158 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.863113    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h952v" event={"ID":"b6fe25b0-03dc-434d-957b-5f25c8d5c141","Type":"ContainerStarted","Data":"a24e27a43277303c542a3fab007c981392e4c6cb4eca6bb9dbef0a265fbdc3c5"}
Nov 03 20:38:45.864395 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.864366    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9hqmv" event={"ID":"e658bf30-634f-4db1-9b41-280a746540a8","Type":"ContainerStarted","Data":"124bfbdf1e4804f9ba9772a37c4c7a844ecefce618897fbf97094e9f69883a9b"}
Nov 03 20:38:45.866017 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.865993    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4dwkb" event={"ID":"e5e0251d-3ac4-4b42-9175-39ffd668896f","Type":"ContainerStarted","Data":"264d7531f8500f215d53e74449911b84b203f7bbeb4549e3f9ab721c1d5ce85f"}
Nov 03 20:38:45.867646 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.867607    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pbhv8" event={"ID":"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43","Type":"ContainerStarted","Data":"d27d19c6586f35ebee131ffe2dc27cd671d13753bdf7c0c7e950c8f166acbf38"}
Nov 03 20:38:45.868814 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.868787    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h" event={"ID":"32cf1fce-f61e-4870-aff9-a9171e90b09f","Type":"ContainerStarted","Data":"62995e99c79deae78ab49a3f98e520833e6ff056e23073fe3b09ddfc5f8a8e5d"}
Nov 03 20:38:45.937480 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.937444    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 20:33:44 +0000 UTC" deadline="2028-06-12 17:20:51.68842727 +0000 UTC"
Nov 03 20:38:45.937480 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:45.937479    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14084h42m5.750952828s"
Nov 03 20:38:46.437252 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:46.437200    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:46.437438 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:46.437298    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:46.437438 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:46.437408    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:46.437555 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:46.437464    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs podName:57913479-90b9-4db4-90f1-81cb81c2e5bb nodeName:}" failed. No retries permitted until 2026-11-03 20:38:48.437445468 +0000 UTC m=+6.172355328 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs") pod "network-metrics-daemon-hmnm7" (UID: "57913479-90b9-4db4-90f1-81cb81c2e5bb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:46.437555 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:46.437525    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:46.437698 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:46.437558    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret podName:9353a680-00bf-4d3a-8437-5a959a40a682 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:48.437547991 +0000 UTC m=+6.172457849 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret") pod "global-pull-secret-syncer-8jhcx" (UID: "9353a680-00bf-4d3a-8437-5a959a40a682") : object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:46.538119 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:46.538077    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhxc\" (UniqueName: \"kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc\") pod \"network-check-target-l9p4t\" (UID: \"6b982774-43c6-45ad-a3e3-77b7bc3d534e\") " pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:38:46.538366 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:46.538341    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 20:38:46.538458 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:46.538397    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 20:38:46.538458 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:46.538415    2575 projected.go:196] Error preparing data for projected volume kube-api-access-5lhxc for pod openshift-network-diagnostics/network-check-target-l9p4t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:46.538553 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:46.538477    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc podName:6b982774-43c6-45ad-a3e3-77b7bc3d534e nodeName:}" failed. No retries permitted until 2026-11-03 20:38:48.538458716 +0000 UTC m=+6.273368573 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5lhxc" (UniqueName: "kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc") pod "network-check-target-l9p4t" (UID: "6b982774-43c6-45ad-a3e3-77b7bc3d534e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:46.852979 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:46.852835    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:38:46.852979 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:46.852964    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:38:46.853186 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:46.853061    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:46.853186 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:46.853156    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:38:46.853280 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:46.853217    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:46.853327 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:46.853288    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:38:46.937737 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:46.937676    2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 20:33:44 +0000 UTC" deadline="2028-04-24 19:19:59.571070587 +0000 UTC"
Nov 03 20:38:46.937737 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:46.937735    2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12910h41m12.633339595s"
Nov 03 20:38:47.891211 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:47.889968    2575 generic.go:356] "Generic (PLEG): container finished" podID="61bc8eba7840edd90e06bc583d329627" containerID="93682a4a249d3236e42950bc5d86fcc9d047fd28b75663925647015447c9e425" exitCode=0
Nov 03 20:38:47.891211 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:47.890241    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal" event={"ID":"61bc8eba7840edd90e06bc583d329627","Type":"ContainerDied","Data":"93682a4a249d3236e42950bc5d86fcc9d047fd28b75663925647015447c9e425"}
Nov 03 20:38:48.454396 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:48.454005    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:48.454396 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:48.454095    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:48.454396 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:48.454210    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:48.454396 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:48.454273    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs podName:57913479-90b9-4db4-90f1-81cb81c2e5bb nodeName:}" failed. No retries permitted until 2026-11-03 20:38:52.454245163 +0000 UTC m=+10.189155034 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs") pod "network-metrics-daemon-hmnm7" (UID: "57913479-90b9-4db4-90f1-81cb81c2e5bb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:48.454396 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:48.454335    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:48.454396 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:48.454365    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret podName:9353a680-00bf-4d3a-8437-5a959a40a682 nodeName:}" failed. No retries permitted until 2026-11-03 20:38:52.454356938 +0000 UTC m=+10.189266793 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret") pod "global-pull-secret-syncer-8jhcx" (UID: "9353a680-00bf-4d3a-8437-5a959a40a682") : object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:48.554946 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:48.554916    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhxc\" (UniqueName: \"kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc\") pod \"network-check-target-l9p4t\" (UID: \"6b982774-43c6-45ad-a3e3-77b7bc3d534e\") " pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:38:48.556056 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:48.555066    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 20:38:48.556056 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:48.555088    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 20:38:48.556056 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:48.555103    2575 projected.go:196] Error preparing data for projected volume kube-api-access-5lhxc for pod openshift-network-diagnostics/network-check-target-l9p4t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:48.556056 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:48.555148    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc podName:6b982774-43c6-45ad-a3e3-77b7bc3d534e nodeName:}" failed. No retries permitted until 2026-11-03 20:38:52.555136919 +0000 UTC m=+10.290046774 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5lhxc" (UniqueName: "kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc") pod "network-check-target-l9p4t" (UID: "6b982774-43c6-45ad-a3e3-77b7bc3d534e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:48.852739 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:48.852705    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:48.852890 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:48.852862    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:38:48.852948 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:48.852922    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:48.853016 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:48.852988    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:38:48.853056 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:48.853024    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:38:48.853134 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:48.853083    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:38:48.895076 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:48.895042    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal" event={"ID":"88d138da692bc0e2c453d9d2e3886bfc","Type":"ContainerStarted","Data":"98e4b4699033c16352e758d565a30a64fbc9b98a10883c80e7223fc8fc21d6b1"}
Nov 03 20:38:48.900944 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:48.900879    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal" event={"ID":"61bc8eba7840edd90e06bc583d329627","Type":"ContainerStarted","Data":"eaa33074687f321d51423a81a98a26e966dcf6222407d8b73e06680d942001fd"}
Nov 03 20:38:48.911868 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:48.911779    2575 generic.go:356] "Generic (PLEG): container finished" podID="0ffcf342-bccb-4362-b79e-8faf3d337b17" containerID="61c7c63d69cd5233d52a1985b96ddaa16a3bb3d58b9a098a3ffbddd343a094fd" exitCode=0
Nov 03 20:38:48.911868 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:48.911820    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxzlf" event={"ID":"0ffcf342-bccb-4362-b79e-8faf3d337b17","Type":"ContainerDied","Data":"61c7c63d69cd5233d52a1985b96ddaa16a3bb3d58b9a098a3ffbddd343a094fd"}
Nov 03 20:38:49.160192 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:49.160061    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal" podStartSLOduration=5.160044846 podStartE2EDuration="5.160044846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:49.032486489 +0000 UTC m=+6.767396367" watchObservedRunningTime="2026-11-03 20:38:49.160044846 +0000 UTC m=+6.894954724"
Nov 03 20:38:49.918218 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:49.918134    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-h952v" event={"ID":"b6fe25b0-03dc-434d-957b-5f25c8d5c141","Type":"ContainerStarted","Data":"7a6b41e810df3a0e6a52ff6f9b17077ecd79dde30d30dd22b9cdd086a2c49489"}
Nov 03 20:38:50.031558 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:50.031495    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal" podStartSLOduration=6.031474561 podStartE2EDuration="6.031474561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:49.275082298 +0000 UTC m=+7.009992177" watchObservedRunningTime="2026-11-03 20:38:50.031474561 +0000 UTC m=+7.766384442"
Nov 03 20:38:50.031789 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:50.031754    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-h952v" podStartSLOduration=2.6356126829999997 podStartE2EDuration="7.031746371s" totalImagesPullingTime="4.396133688s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:50.03169844 +0000 UTC m=+7.766608309" watchObservedRunningTime="2026-11-03 20:38:50.031746371 +0000 UTC m=+7.766656251"
Nov 03 20:38:50.851103 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:50.850799    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:50.851103 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:50.850927    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:38:50.851103 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:50.850997    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:38:50.851103 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:50.851061    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:38:50.851487 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:50.851467    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:50.851596 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:50.851576    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:38:50.924439 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:50.924322    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9hqmv" event={"ID":"e658bf30-634f-4db1-9b41-280a746540a8","Type":"ContainerStarted","Data":"9a70ee94b18edd6dbffd28eddce4bab1aa3aa38aaf5bbea99a6185830831da48"}
Nov 03 20:38:51.930827 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:51.930747    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h" event={"ID":"32cf1fce-f61e-4870-aff9-a9171e90b09f","Type":"ContainerStarted","Data":"ebd9b9fad7bbbeeb1be19b07efc65a1060b0c0fda697013a5eb59d8169a4eb2b"}
Nov 03 20:38:52.488818 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:52.488739    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:52.488983 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:52.488808    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:52.489059 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:52.488979    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:52.489059 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:52.489039    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret podName:9353a680-00bf-4d3a-8437-5a959a40a682 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:00.489021866 +0000 UTC m=+18.223931720 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret") pod "global-pull-secret-syncer-8jhcx" (UID: "9353a680-00bf-4d3a-8437-5a959a40a682") : object "kube-system"/"original-pull-secret" not registered
Nov 03 20:38:52.489460 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:52.489431    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:52.489579 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:52.489490    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs podName:57913479-90b9-4db4-90f1-81cb81c2e5bb nodeName:}" failed. No retries permitted until 2026-11-03 20:39:00.489474867 +0000 UTC m=+18.224384721 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs") pod "network-metrics-daemon-hmnm7" (UID: "57913479-90b9-4db4-90f1-81cb81c2e5bb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:38:52.591235 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:52.590928    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhxc\" (UniqueName: \"kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc\") pod \"network-check-target-l9p4t\" (UID: \"6b982774-43c6-45ad-a3e3-77b7bc3d534e\") " pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:38:52.591235 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:52.591099    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 20:38:52.591235 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:52.591121    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 20:38:52.591235 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:52.591135    2575 projected.go:196] Error preparing data for projected volume kube-api-access-5lhxc for pod openshift-network-diagnostics/network-check-target-l9p4t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:52.591235 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:52.591195    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc podName:6b982774-43c6-45ad-a3e3-77b7bc3d534e nodeName:}" failed. No retries permitted until 2026-11-03 20:39:00.591177229 +0000 UTC m=+18.326087110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5lhxc" (UniqueName: "kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc") pod "network-check-target-l9p4t" (UID: "6b982774-43c6-45ad-a3e3-77b7bc3d534e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:38:52.851677 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:52.851587    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:52.852189 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:52.852131    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:38:52.852360 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:52.851987    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:52.852545 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:52.852528    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:38:52.852662 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:52.851907    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:38:52.852806 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:52.852788    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:38:52.895524 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:52.894979    2575 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:38:52.896446 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:52.896176    2575 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:38:53.014647 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:53.014541    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9hqmv" podStartSLOduration=5.504510554 podStartE2EDuration="11.014523101s" totalImagesPullingTime="5.510012547s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:51.03794352 +0000 UTC m=+8.772853397" watchObservedRunningTime="2026-11-03 20:38:53.014523101 +0000 UTC m=+10.749432979"
Nov 03 20:38:53.689990 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:53.689958    2575 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 03 20:38:53.795453 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:53.795341    2575 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T20:38:53.689979361Z","UUID":"60e0d18d-ff35-4430-a3d2-3abcb42452e5","Handler":null,"Name":"","Endpoint":""}
Nov 03 20:38:53.797221 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:53.797164    2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 03 20:38:53.797221 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:53.797189    2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 03 20:38:53.939481 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:53.939450    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h" event={"ID":"32cf1fce-f61e-4870-aff9-a9171e90b09f","Type":"ContainerStarted","Data":"26cc3232235e272967e244300c03dd539d926c9e0aa84ce10f952fd56473d637"}
Nov 03 20:38:53.942022 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:53.941938    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxzlf" event={"ID":"0ffcf342-bccb-4362-b79e-8faf3d337b17","Type":"ContainerStarted","Data":"264da65c804b5693f6c000f725034c2aa12e387ca7284e7ed7426ac0155d1e63"}
Nov 03 20:38:53.944393 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:53.944368    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-585md" event={"ID":"10a65b13-a9e4-4355-8bac-fab245388f16","Type":"ContainerStarted","Data":"d16da64ef07bf8ae9de2acd6a99ae677e694e805d55ac6db44c5a748696f8706"}
Nov 03 20:38:54.178767 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:54.178549    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-585md" podStartSLOduration=3.133191941 podStartE2EDuration="11.178532445s" totalImagesPullingTime="8.045340504s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:54.177835373 +0000 UTC m=+11.912745251" watchObservedRunningTime="2026-11-03 20:38:54.178532445 +0000 UTC m=+11.913442322"
Nov 03 20:38:54.850841 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:54.850780    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:38:54.850841 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:54.850806    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:54.851067 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:54.850881    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:54.851067 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:54.850983    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:38:54.851174 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:54.851112    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:38:54.851228 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:54.851206    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:38:54.948526 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:54.948285    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4dwkb" event={"ID":"e5e0251d-3ac4-4b42-9175-39ffd668896f","Type":"ContainerStarted","Data":"b4f1829a2c5980e00da5a801347ca8dbe2d815238f4aa026ce66d079a1ad9d9d"}
Nov 03 20:38:54.950870 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:54.950811    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pbhv8" event={"ID":"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43","Type":"ContainerStarted","Data":"f7c95a4068bf65b14b2f9f21b54455d631c71452565fc2d06e24197dab7902c9"}
Nov 03 20:38:54.953745 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:54.953694    2575 generic.go:356] "Generic (PLEG): container finished" podID="0ffcf342-bccb-4362-b79e-8faf3d337b17" containerID="264da65c804b5693f6c000f725034c2aa12e387ca7284e7ed7426ac0155d1e63" exitCode=0
Nov 03 20:38:54.954668 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:54.954619    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxzlf" event={"ID":"0ffcf342-bccb-4362-b79e-8faf3d337b17","Type":"ContainerDied","Data":"264da65c804b5693f6c000f725034c2aa12e387ca7284e7ed7426ac0155d1e63"}
Nov 03 20:38:55.064157 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:55.064082    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4dwkb" podStartSLOduration=3.738590352 podStartE2EDuration="12.064065662s" totalImagesPullingTime="8.32547531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:55.063749193 +0000 UTC m=+12.798659095" watchObservedRunningTime="2026-11-03 20:38:55.064065662 +0000 UTC m=+12.798975542"
Nov 03 20:38:55.178497 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:55.178453    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-pbhv8" podStartSLOduration=3.517544913 podStartE2EDuration="12.178438464s" totalImagesPullingTime="8.660893551s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:55.178312243 +0000 UTC m=+12.913222123" watchObservedRunningTime="2026-11-03 20:38:55.178438464 +0000 UTC m=+12.913348340"
Nov 03 20:38:55.958859 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:55.958829    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h" event={"ID":"32cf1fce-f61e-4870-aff9-a9171e90b09f","Type":"ContainerStarted","Data":"0ef2074c567e0c38c7589755f23e2ffaa2c55e5a822d513b72afd971bd28390d"}
Nov 03 20:38:56.076168 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:56.076121    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h" podStartSLOduration=3.559748189 podStartE2EDuration="13.076105153s" totalImagesPullingTime="9.516356964s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:56.075955923 +0000 UTC m=+13.810865818" watchObservedRunningTime="2026-11-03 20:38:56.076105153 +0000 UTC m=+13.811015030"
Nov 03 20:38:56.850504 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:56.850467    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:56.850709 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:56.850535    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:38:56.850709 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:56.850531    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:56.850709 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:56.850652    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:38:56.850870 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:56.850706    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:38:56.850870 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:56.850827    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:38:56.963072 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:56.963040    2575 generic.go:356] "Generic (PLEG): container finished" podID="0ffcf342-bccb-4362-b79e-8faf3d337b17" containerID="6f84edd7aec47fab4920eb5c23965afdaec9c42e9f01dfd3b8d930a0e00a0862" exitCode=0
Nov 03 20:38:56.963671 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:56.963131    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxzlf" event={"ID":"0ffcf342-bccb-4362-b79e-8faf3d337b17","Type":"ContainerDied","Data":"6f84edd7aec47fab4920eb5c23965afdaec9c42e9f01dfd3b8d930a0e00a0862"}
Nov 03 20:38:56.964883 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:56.964853    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dm2zp" event={"ID":"1169f5bf-4ec8-4f85-aba9-857c3a666a3f","Type":"ContainerStarted","Data":"d9251c7a928810d1eeba1f3fa5fd266f318448c4d998596fc6546dfb50fff92f"}
Nov 03 20:38:57.300844 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:57.300754    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dm2zp" podStartSLOduration=3.521488452 podStartE2EDuration="14.300734135s" totalImagesPullingTime="10.779245683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:38:57.300234617 +0000 UTC m=+15.035144494" watchObservedRunningTime="2026-11-03 20:38:57.300734135 +0000 UTC m=+15.035644013"
Nov 03 20:38:58.850865 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:58.850673    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:38:58.851365 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:58.850672    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:38:58.851365 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:58.850958    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:38:58.851365 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:58.850712    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:38:58.851365 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:58.851068    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:38:58.851365 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:38:58.851112    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:38:58.971592 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:58.971561    2575 generic.go:356] "Generic (PLEG): container finished" podID="0ffcf342-bccb-4362-b79e-8faf3d337b17" containerID="46430392033ef2e3c328d1a3386bf50e81610fb104d080ac7eb5ad4e340c39cd" exitCode=0
Nov 03 20:38:58.971755 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:38:58.971620    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxzlf" event={"ID":"0ffcf342-bccb-4362-b79e-8faf3d337b17","Type":"ContainerDied","Data":"46430392033ef2e3c328d1a3386bf50e81610fb104d080ac7eb5ad4e340c39cd"}
Nov 03 20:39:00.551862 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:00.551599    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:00.552270 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:00.551889    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:00.552270 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:00.551719    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:39:00.552270 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:00.551983    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs podName:57913479-90b9-4db4-90f1-81cb81c2e5bb nodeName:}" failed. No retries permitted until 2026-11-03 20:39:16.551965732 +0000 UTC m=+34.286875587 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs") pod "network-metrics-daemon-hmnm7" (UID: "57913479-90b9-4db4-90f1-81cb81c2e5bb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:39:00.552270 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:00.552007    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 20:39:00.552270 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:00.552062    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret podName:9353a680-00bf-4d3a-8437-5a959a40a682 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:16.552047085 +0000 UTC m=+34.286956954 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret") pod "global-pull-secret-syncer-8jhcx" (UID: "9353a680-00bf-4d3a-8437-5a959a40a682") : object "kube-system"/"original-pull-secret" not registered
Nov 03 20:39:00.652728 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:00.652646    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhxc\" (UniqueName: \"kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc\") pod \"network-check-target-l9p4t\" (UID: \"6b982774-43c6-45ad-a3e3-77b7bc3d534e\") " pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:00.652863 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:00.652813    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 20:39:00.652863 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:00.652834    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 20:39:00.652863 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:00.652844    2575 projected.go:196] Error preparing data for projected volume kube-api-access-5lhxc for pod openshift-network-diagnostics/network-check-target-l9p4t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:39:00.652990 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:00.652900    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc podName:6b982774-43c6-45ad-a3e3-77b7bc3d534e nodeName:}" failed. No retries permitted until 2026-11-03 20:39:16.652884003 +0000 UTC m=+34.387793878 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5lhxc" (UniqueName: "kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc") pod "network-check-target-l9p4t" (UID: "6b982774-43c6-45ad-a3e3-77b7bc3d534e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:39:00.850265 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:00.850226    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:00.850265 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:00.850253    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:00.850449 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:00.850300    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:00.850449 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:00.850411    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:00.850536 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:00.850517    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:00.850610 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:00.850594    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:00.979160 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:00.979129    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf" event={"ID":"11516670-b4cd-43cc-8c41-64e5bfed2b5a","Type":"ContainerStarted","Data":"53e74d4716835940f6c772196283958836250a3a3728df17453a7319886dcf4d"}
Nov 03 20:39:00.979299 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:00.979210    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf" event={"ID":"11516670-b4cd-43cc-8c41-64e5bfed2b5a","Type":"ContainerStarted","Data":"3c26427352e42a98a565e8b2f27e08b8cb782cddd4f486e33ca91f913e838406"}
Nov 03 20:39:00.979299 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:00.979242    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf" event={"ID":"11516670-b4cd-43cc-8c41-64e5bfed2b5a","Type":"ContainerStarted","Data":"7b37baf064f3da07b483d9b4b61a585de63f3d0fe5efed1236aeb20ab80d9cc2"}
Nov 03 20:39:00.979375 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:00.979319    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf" event={"ID":"11516670-b4cd-43cc-8c41-64e5bfed2b5a","Type":"ContainerStarted","Data":"f3ceb95f731d30ad93941651380c66429c252193163fc23e1ca9936d384fc5e3"}
Nov 03 20:39:00.979375 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:00.979354    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf" event={"ID":"11516670-b4cd-43cc-8c41-64e5bfed2b5a","Type":"ContainerStarted","Data":"a530bd4e35360b83ee869a6fd9ab59827e349851d69ff740517f6158e20269f2"}
Nov 03 20:39:00.979479 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:00.979388    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf" event={"ID":"11516670-b4cd-43cc-8c41-64e5bfed2b5a","Type":"ContainerStarted","Data":"975ebdc6e445b97c52a0d66d70ce598168017a8ad8cd4c074e8812c20bcdee16"}
Nov 03 20:39:00.982006 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:00.981985    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxzlf" event={"ID":"0ffcf342-bccb-4362-b79e-8faf3d337b17","Type":"ContainerStarted","Data":"e84b98cd65ad35bfa8543281d6908a65963eaaafbf82050e65f4124448cb257b"}
Nov 03 20:39:01.986798 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:01.986757    2575 generic.go:356] "Generic (PLEG): container finished" podID="0ffcf342-bccb-4362-b79e-8faf3d337b17" containerID="e84b98cd65ad35bfa8543281d6908a65963eaaafbf82050e65f4124448cb257b" exitCode=0
Nov 03 20:39:01.987256 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:01.986821    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxzlf" event={"ID":"0ffcf342-bccb-4362-b79e-8faf3d337b17","Type":"ContainerDied","Data":"e84b98cd65ad35bfa8543281d6908a65963eaaafbf82050e65f4124448cb257b"}
Nov 03 20:39:02.851451 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:02.851370    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:02.851679 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:02.851481    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:02.851679 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:02.851578    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:02.851803 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:02.851703    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:02.851803 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:02.851761    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:02.851878 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:02.851857    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:02.991782 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:02.991748    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf" event={"ID":"11516670-b4cd-43cc-8c41-64e5bfed2b5a","Type":"ContainerStarted","Data":"6efa7ffd4b77db751ec5dd97a481589ef13a49f8f0941e2fbda3ae2bf930d4b8"}
Nov 03 20:39:02.994870 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:02.994842    2575 generic.go:356] "Generic (PLEG): container finished" podID="0ffcf342-bccb-4362-b79e-8faf3d337b17" containerID="99bcd51319ec2acd93b17eb9747949fb2230cd212c3ee200a3155c813661b322" exitCode=0
Nov 03 20:39:02.994975 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:02.994875    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxzlf" event={"ID":"0ffcf342-bccb-4362-b79e-8faf3d337b17","Type":"ContainerDied","Data":"99bcd51319ec2acd93b17eb9747949fb2230cd212c3ee200a3155c813661b322"}
Nov 03 20:39:03.999930 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:03.999896    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lxzlf" event={"ID":"0ffcf342-bccb-4362-b79e-8faf3d337b17","Type":"ContainerStarted","Data":"0d2999626209d03569031137ef62e772ab5b6185d5eedcf3aee21aeec66cdd41"}
Nov 03 20:39:04.219783 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:04.219735    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lxzlf" podStartSLOduration=8.729902868 podStartE2EDuration="21.219722708s" totalImagesPullingTime="12.48981984s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:04.219272621 +0000 UTC m=+21.954182498" watchObservedRunningTime="2026-11-03 20:39:04.219722708 +0000 UTC m=+21.954632595"
Nov 03 20:39:04.851150 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:04.850911    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:04.851275 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:04.850937    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:04.851275 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:04.851173    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:04.851275 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:04.851244    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:04.851275 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:04.850981    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:04.851409 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:04.851315    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:05.005187 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:05.005150    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf" event={"ID":"11516670-b4cd-43cc-8c41-64e5bfed2b5a","Type":"ContainerStarted","Data":"9db378d04d49034322e2d259db5cb6eb4a093e315d685da7063eb297cd50274a"}
Nov 03 20:39:05.184190 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:05.184098    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf" podStartSLOduration=7.272811181 podStartE2EDuration="22.184085429s" totalImagesPullingTime="14.911274248s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:05.182939764 +0000 UTC m=+22.917849640" watchObservedRunningTime="2026-11-03 20:39:05.184085429 +0000 UTC m=+22.918995303"
Nov 03 20:39:06.007529 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:06.007494    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:06.008004 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:06.007617    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:06.008004 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:06.007686    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:06.023273 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:06.023251    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:06.023473 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:06.023459    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:06.850175 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:06.850149    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:06.850341 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:06.850251    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:06.850341 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:06.850263    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:06.850341 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:06.850282    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:06.850449 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:06.850401    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:06.850510 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:06.850491    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:07.310666 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:07.310580    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8jhcx"]
Nov 03 20:39:07.311130 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:07.310691    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:07.311130 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:07.310775    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:07.314235 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:07.314205    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l9p4t"]
Nov 03 20:39:07.314354 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:07.314301    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:07.314423 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:07.314402    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:07.314979 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:07.314957    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hmnm7"]
Nov 03 20:39:07.315084 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:07.315047    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:07.315138 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:07.315127    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:08.850621 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:08.850587    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:08.851069 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:08.850592    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:08.851069 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:08.850726    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:08.851069 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:08.850724    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:08.851069 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:08.850783    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:08.851069 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:08.850877    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:10.010971 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:10.010903    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:39:10.011393 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:10.011022    2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 20:39:10.011444 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:10.011401    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:39:10.850477 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:10.850444    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:10.850588 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:10.850500    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:10.850588 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:10.850502    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:10.850698 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:10.850582    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:10.850780 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:10.850749    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:10.850882 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:10.850849    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:12.850963 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:12.850928    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:12.854288 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:12.851420    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:12.854288 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:12.851499    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:12.854288 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:12.851643    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:12.854288 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:12.851855    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:12.854288 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:12.852117    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:14.851212 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:14.851033    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:14.851612 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:14.851074    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:14.851612 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:14.851287    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:14.851612 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:14.851074    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:14.851612 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:14.851384    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:14.851612 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:14.851447    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:16.574018 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:16.573985    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:16.574370 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:16.574048    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:16.574370 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:16.574122    2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 03 20:39:16.574370 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:16.574137    2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:39:16.574370 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:16.574178    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret podName:9353a680-00bf-4d3a-8437-5a959a40a682 nodeName:}" failed. No retries permitted until 2026-11-03 20:39:48.574159392 +0000 UTC m=+66.309069263 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret") pod "global-pull-secret-syncer-8jhcx" (UID: "9353a680-00bf-4d3a-8437-5a959a40a682") : object "kube-system"/"original-pull-secret" not registered
Nov 03 20:39:16.574370 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:16.574192    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs podName:57913479-90b9-4db4-90f1-81cb81c2e5bb nodeName:}" failed. No retries permitted until 2026-11-03 20:39:48.574185775 +0000 UTC m=+66.309095630 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs") pod "network-metrics-daemon-hmnm7" (UID: "57913479-90b9-4db4-90f1-81cb81c2e5bb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 03 20:39:16.675213 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:16.675188    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhxc\" (UniqueName: \"kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc\") pod \"network-check-target-l9p4t\" (UID: \"6b982774-43c6-45ad-a3e3-77b7bc3d534e\") " pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:16.675349 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:16.675305    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 03 20:39:16.675349 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:16.675317    2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 03 20:39:16.675349 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:16.675326    2575 projected.go:196] Error preparing data for projected volume kube-api-access-5lhxc for pod openshift-network-diagnostics/network-check-target-l9p4t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:39:16.675469 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:16.675364    2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc podName:6b982774-43c6-45ad-a3e3-77b7bc3d534e nodeName:}" failed. No retries permitted until 2026-11-03 20:39:48.675353335 +0000 UTC m=+66.410263190 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5lhxc" (UniqueName: "kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc") pod "network-check-target-l9p4t" (UID: "6b982774-43c6-45ad-a3e3-77b7bc3d534e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 03 20:39:16.850111 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:16.850049    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:16.850233 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:16.850145    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:16.850233 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:16.850059    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:16.850337 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:16.850240    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:16.850337 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:16.850053    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:16.850429 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:16.850370    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:18.849968 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:18.849937    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:18.850363 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:18.850032    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:18.850363 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:18.850073    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:18.850363 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:18.850146    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:18.850363 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:18.850169    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:18.850363 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:18.850228    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:19.439239 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:19.439212    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pbhv8_853d1e90-3b6e-4bb5-b6c5-82b1ce141e43/dns-node-resolver/0.log"
Nov 03 20:39:19.974486 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:19.974460    2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9hqmv_e658bf30-634f-4db1-9b41-280a746540a8/node-ca/0.log"
Nov 03 20:39:20.849998 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:20.849965    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:20.849998 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:20.849990    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:20.850230 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:20.850093    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:20.850230 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:20.850170    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:20.851845 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:20.850671    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:20.851845 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:20.850294    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:22.853351 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:22.853317    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:22.853351 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:22.853344    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:22.853841 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:22.853397    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:22.853841 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:22.853319    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:22.853841 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:22.853499    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:22.853841 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:22.853599    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:24.850024 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:24.849995    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:24.850024 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:24.850019    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:24.850523 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:24.850127    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:24.850523 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:24.850170    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:24.850523 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:24.850215    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:24.850523 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:24.850275    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:26.850734 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:26.850701    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:26.851255 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:26.850701    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:26.851255 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:26.850826    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:26.851255 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:26.850701    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:26.851255 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:26.850899    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:26.851255 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:26.851021    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:28.849958 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:28.849923    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:28.850448 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:28.849927    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:28.850448 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:28.850031    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:28.850448 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:28.849927    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:28.850448 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:28.850100    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:28.850448 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:28.850170    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:30.850735 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:30.850702    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:30.851212 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:30.850711    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:30.851212 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:30.850817    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:30.851212 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:30.850910    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:30.851212 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:30.850970    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:30.851212 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:30.851047    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:32.851321 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:32.851287    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:32.851759 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:32.851342    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:32.851759 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:32.851411    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:32.851759 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:32.851435    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:32.851759 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:32.851534    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:32.851759 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:32.851620    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:34.850841 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:34.850799    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:34.850841 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:34.850817    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:34.851382 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:34.850904    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:34.851382 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:34.850920    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:34.851382 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:34.851004    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:34.851382 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:34.851067    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:36.850649 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:36.850606    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:36.851106 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:36.850684    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:36.851106 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:36.850783    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:36.851106 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:36.850829    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:36.851106 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:36.850915    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:36.851106 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:36.850984    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:38.021302 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:38.021272    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:38.850414 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:38.850385    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:38.850599 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:38.850486    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hmnm7" podUID="57913479-90b9-4db4-90f1-81cb81c2e5bb"
Nov 03 20:39:38.850599 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:38.850385    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:38.850599 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:38.850525    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:38.850599 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:38.850583    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l9p4t" podUID="6b982774-43c6-45ad-a3e3-77b7bc3d534e"
Nov 03 20:39:38.850842 ip-10-0-142-223 kubenswrapper[2575]: E1103 20:39:38.850669    2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8jhcx" podUID="9353a680-00bf-4d3a-8437-5a959a40a682"
Nov 03 20:39:40.072531 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.072504    2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeReady"
Nov 03 20:39:40.072887 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.072603    2575 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 03 20:39:40.237283 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.237258    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-45x6l"]
Nov 03 20:39:40.239202 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.239184    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kmzjf"]
Nov 03 20:39:40.239308 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.239294    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:40.240831 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.240815    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-j86dg"]
Nov 03 20:39:40.240973 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.240952    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kmzjf"
Nov 03 20:39:40.242561 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.242543    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-45x6l"]
Nov 03 20:39:40.242664 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.242652    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.260010 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.259992    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j86dg"]
Nov 03 20:39:40.291509 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.291476    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 03 20:39:40.291723 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.291709    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9ll5z\""
Nov 03 20:39:40.291787 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.291713    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 03 20:39:40.292704 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.292691    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 03 20:39:40.292792 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.292777    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 03 20:39:40.299955 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.299921    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kmzjf"]
Nov 03 20:39:40.342457 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.342432    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/346aebd9-354c-4683-9eff-58058e9919c7-crio-socket\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.342571 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.342460    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/131affdb-a6b8-457d-84b6-ffceca9f2bfd-cert\") pod \"ingress-canary-kmzjf\" (UID: \"131affdb-a6b8-457d-84b6-ffceca9f2bfd\") " pod="openshift-ingress-canary/ingress-canary-kmzjf"
Nov 03 20:39:40.342571 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.342482    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-metrics-tls\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:40.342571 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.342564    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/346aebd9-354c-4683-9eff-58058e9919c7-data-volume\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.342753 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.342586    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/346aebd9-354c-4683-9eff-58058e9919c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.342753 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.342603    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f6kw\" (UniqueName: \"kubernetes.io/projected/346aebd9-354c-4683-9eff-58058e9919c7-kube-api-access-2f6kw\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.342753 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.342647    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-tmp-dir\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:40.342753 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.342707    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrcsl\" (UniqueName: \"kubernetes.io/projected/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-kube-api-access-nrcsl\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:40.342896 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.342808    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-config-volume\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:40.342896 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.342850    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkzsl\" (UniqueName: \"kubernetes.io/projected/131affdb-a6b8-457d-84b6-ffceca9f2bfd-kube-api-access-hkzsl\") pod \"ingress-canary-kmzjf\" (UID: \"131affdb-a6b8-457d-84b6-ffceca9f2bfd\") " pod="openshift-ingress-canary/ingress-canary-kmzjf"
Nov 03 20:39:40.342896 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.342870    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/346aebd9-354c-4683-9eff-58058e9919c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.392720 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.392685    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 03 20:39:40.392854 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.392750    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 03 20:39:40.392854 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.392753    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 03 20:39:40.392854 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.392805    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-fjpbj\""
Nov 03 20:39:40.392854 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.392829    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 03 20:39:40.392854 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.392845    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9fj5k\""
Nov 03 20:39:40.393041 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.392898    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 03 20:39:40.443737 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.443713    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/131affdb-a6b8-457d-84b6-ffceca9f2bfd-cert\") pod \"ingress-canary-kmzjf\" (UID: \"131affdb-a6b8-457d-84b6-ffceca9f2bfd\") " pod="openshift-ingress-canary/ingress-canary-kmzjf"
Nov 03 20:39:40.443832 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.443744    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-metrics-tls\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:40.443832 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.443777    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/346aebd9-354c-4683-9eff-58058e9919c7-data-volume\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.443922 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.443904    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/346aebd9-354c-4683-9eff-58058e9919c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.443971 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.443937    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f6kw\" (UniqueName: \"kubernetes.io/projected/346aebd9-354c-4683-9eff-58058e9919c7-kube-api-access-2f6kw\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.443971 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.443956    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-tmp-dir\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:40.444073 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.443979    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrcsl\" (UniqueName: \"kubernetes.io/projected/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-kube-api-access-nrcsl\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:40.444073 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.444023    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-config-volume\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:40.444179 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.444075    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkzsl\" (UniqueName: \"kubernetes.io/projected/131affdb-a6b8-457d-84b6-ffceca9f2bfd-kube-api-access-hkzsl\") pod \"ingress-canary-kmzjf\" (UID: \"131affdb-a6b8-457d-84b6-ffceca9f2bfd\") " pod="openshift-ingress-canary/ingress-canary-kmzjf"
Nov 03 20:39:40.444179 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.444099    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/346aebd9-354c-4683-9eff-58058e9919c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.444179 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.444108    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/346aebd9-354c-4683-9eff-58058e9919c7-data-volume\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.444179 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.444127    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/346aebd9-354c-4683-9eff-58058e9919c7-crio-socket\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.444379 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.444300    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/346aebd9-354c-4683-9eff-58058e9919c7-crio-socket\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.444379 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.444314    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-tmp-dir\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:40.444577 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.444558    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/346aebd9-354c-4683-9eff-58058e9919c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.444685 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.444666    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-config-volume\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:40.447708 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.447689    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-metrics-tls\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:40.447791 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.447713    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/346aebd9-354c-4683-9eff-58058e9919c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.447791 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.447722    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/131affdb-a6b8-457d-84b6-ffceca9f2bfd-cert\") pod \"ingress-canary-kmzjf\" (UID: \"131affdb-a6b8-457d-84b6-ffceca9f2bfd\") " pod="openshift-ingress-canary/ingress-canary-kmzjf"
Nov 03 20:39:40.504206 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.504150    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrcsl\" (UniqueName: \"kubernetes.io/projected/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-kube-api-access-nrcsl\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:40.504417 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.504398    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f6kw\" (UniqueName: \"kubernetes.io/projected/346aebd9-354c-4683-9eff-58058e9919c7-kube-api-access-2f6kw\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.504615 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.504595    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkzsl\" (UniqueName: \"kubernetes.io/projected/131affdb-a6b8-457d-84b6-ffceca9f2bfd-kube-api-access-hkzsl\") pod \"ingress-canary-kmzjf\" (UID: \"131affdb-a6b8-457d-84b6-ffceca9f2bfd\") " pod="openshift-ingress-canary/ingress-canary-kmzjf"
Nov 03 20:39:40.552012 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.551975    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:40.557394 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.557380    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kmzjf"
Nov 03 20:39:40.564297 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.564279    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:40.850201 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.850125    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:40.850201 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.850141    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:40.850414 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.850298    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:40.939530 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.939502    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j86dg"]
Nov 03 20:39:40.962957 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.962927    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j2mnh\""
Nov 03 20:39:40.963062 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.963008    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-mtgd7\""
Nov 03 20:39:40.963062 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.963048    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 20:39:40.963062 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.963062    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zvv9n\""
Nov 03 20:39:40.963180 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.963068    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 20:39:40.987713 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.987692    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kmzjf"]
Nov 03 20:39:40.990477 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.990457    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-45x6l"]
Nov 03 20:39:40.991404 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:39:40.991381    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod131affdb_a6b8_457d_84b6_ffceca9f2bfd.slice/crio-e00279b084939ae0376b5e253923ddd2b2b67c4dd139b86e80849df2438b28d5 WatchSource:0}: Error finding container e00279b084939ae0376b5e253923ddd2b2b67c4dd139b86e80849df2438b28d5: Status 404 returned error can't find the container with id e00279b084939ae0376b5e253923ddd2b2b67c4dd139b86e80849df2438b28d5
Nov 03 20:39:40.993364 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:39:40.993337    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cd18396_1fa6_4b85_ad2f_922c2f4a58e2.slice/crio-596edf3c444e2d280e16329d46079439a81596ec44f3107fc1442753f79e2b8d WatchSource:0}: Error finding container 596edf3c444e2d280e16329d46079439a81596ec44f3107fc1442753f79e2b8d: Status 404 returned error can't find the container with id 596edf3c444e2d280e16329d46079439a81596ec44f3107fc1442753f79e2b8d
Nov 03 20:39:40.995785 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.995763    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 20:39:40.995929 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:40.995916    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 20:39:41.074138 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:41.074114    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j86dg" event={"ID":"346aebd9-354c-4683-9eff-58058e9919c7","Type":"ContainerStarted","Data":"528fefa7068330f7531dd4919a81461ddd667e2af87ca04edab435552e7e9f46"}
Nov 03 20:39:41.074486 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:41.074173    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j86dg" event={"ID":"346aebd9-354c-4683-9eff-58058e9919c7","Type":"ContainerStarted","Data":"2da42e99af23864aab781534a38b114b3161276f81da308f8a18052b84a7b741"}
Nov 03 20:39:41.075249 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:41.075227    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kmzjf" event={"ID":"131affdb-a6b8-457d-84b6-ffceca9f2bfd","Type":"ContainerStarted","Data":"e00279b084939ae0376b5e253923ddd2b2b67c4dd139b86e80849df2438b28d5"}
Nov 03 20:39:41.076242 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:41.076224    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-45x6l" event={"ID":"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2","Type":"ContainerStarted","Data":"596edf3c444e2d280e16329d46079439a81596ec44f3107fc1442753f79e2b8d"}
Nov 03 20:39:42.929705 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:42.929660    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt"]
Nov 03 20:39:42.931691 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:42.931664    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt"
Nov 03 20:39:42.962101 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:42.962065    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c4cacc5f-ade7-47d9-af40-78cdc6e090ba-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-gwzjt\" (UID: \"c4cacc5f-ade7-47d9-af40-78cdc6e090ba\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt"
Nov 03 20:39:42.983985 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:42.983958    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Nov 03 20:39:42.984087 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:42.983967    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-sj26z\""
Nov 03 20:39:43.022470 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:43.022397    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt"]
Nov 03 20:39:43.062561 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:43.062475    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c4cacc5f-ade7-47d9-af40-78cdc6e090ba-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-gwzjt\" (UID: \"c4cacc5f-ade7-47d9-af40-78cdc6e090ba\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt"
Nov 03 20:39:43.065605 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:43.065581    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c4cacc5f-ade7-47d9-af40-78cdc6e090ba-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-gwzjt\" (UID: \"c4cacc5f-ade7-47d9-af40-78cdc6e090ba\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt"
Nov 03 20:39:43.085778 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:43.085752    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j86dg" event={"ID":"346aebd9-354c-4683-9eff-58058e9919c7","Type":"ContainerStarted","Data":"af342bed8d2cbdeb7abde31abc858974d610a91bbc039e64d880ce14b1da8b8e"}
Nov 03 20:39:43.244367 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:43.244338    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt"
Nov 03 20:39:43.627114 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:43.627076    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt"]
Nov 03 20:39:43.631125 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:39:43.631097    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4cacc5f_ade7_47d9_af40_78cdc6e090ba.slice/crio-a168d8aade5dd2c4b5cfcf08cbb196fb2e610039d92d4de120ea4f66393a6a03 WatchSource:0}: Error finding container a168d8aade5dd2c4b5cfcf08cbb196fb2e610039d92d4de120ea4f66393a6a03: Status 404 returned error can't find the container with id a168d8aade5dd2c4b5cfcf08cbb196fb2e610039d92d4de120ea4f66393a6a03
Nov 03 20:39:44.091235 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:44.091108    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-45x6l" event={"ID":"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2","Type":"ContainerStarted","Data":"c9d6087772c9ef4b590030215b12b60623820ec604dbbe89922864070c416406"}
Nov 03 20:39:44.091235 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:44.091189    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-45x6l" event={"ID":"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2","Type":"ContainerStarted","Data":"59119e4749a0ffb0519519f4d968adc3647c9d4958f9bfb5e9db89a6d45b7e45"}
Nov 03 20:39:44.091819 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:44.091454    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:44.092708 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:44.092685    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt" event={"ID":"c4cacc5f-ade7-47d9-af40-78cdc6e090ba","Type":"ContainerStarted","Data":"a168d8aade5dd2c4b5cfcf08cbb196fb2e610039d92d4de120ea4f66393a6a03"}
Nov 03 20:39:44.428508 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:44.428412    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-45x6l" podStartSLOduration=1.808579526 podStartE2EDuration="4.428398019s" totalImagesPullingTime="2.619818493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:39:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:44.426988466 +0000 UTC m=+62.161898345" watchObservedRunningTime="2026-11-03 20:39:44.428398019 +0000 UTC m=+62.163307896"
Nov 03 20:39:45.097226 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:45.097198    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j86dg" event={"ID":"346aebd9-354c-4683-9eff-58058e9919c7","Type":"ContainerStarted","Data":"222349dd5debbd0819ad38ab5c4896579f59289dd25cd75103d63eba8e13e311"}
Nov 03 20:39:45.098747 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:45.098724    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kmzjf" event={"ID":"131affdb-a6b8-457d-84b6-ffceca9f2bfd","Type":"ContainerStarted","Data":"1e37a9483f0b9b1eea9ad2832b8439e4a1d86bb682537966d592b77411a1d1d2"}
Nov 03 20:39:45.216813 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:45.216757    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-j86dg" podStartSLOduration=1.8490044879999998 podStartE2EDuration="5.216743418s" totalImagesPullingTime="3.36773893s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 20:39:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:45.215777806 +0000 UTC m=+62.950687683" watchObservedRunningTime="2026-11-03 20:39:45.216743418 +0000 UTC m=+62.951653302"
Nov 03 20:39:45.331393 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:45.331313    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kmzjf" podStartSLOduration=1.812879707 podStartE2EDuration="5.331300621s" totalImagesPullingTime="3.518420914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:39:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:45.330872828 +0000 UTC m=+63.065782716" watchObservedRunningTime="2026-11-03 20:39:45.331300621 +0000 UTC m=+63.066210498"
Nov 03 20:39:46.103237 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:46.103194    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt" event={"ID":"c4cacc5f-ade7-47d9-af40-78cdc6e090ba","Type":"ContainerStarted","Data":"520d8dd53aa6f0db95bf6939598302faaf167ae697cefae763a1e955e42a08c3"}
Nov 03 20:39:46.217591 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:46.217543    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt" podStartSLOduration=2.789774884 podStartE2EDuration="4.217530107s" totalImagesPullingTime="1.427755223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:39:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:46.217080673 +0000 UTC m=+63.951990551" watchObservedRunningTime="2026-11-03 20:39:46.217530107 +0000 UTC m=+63.952439983"
Nov 03 20:39:47.106786 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:47.106748    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt"
Nov 03 20:39:47.111271 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:47.111246    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt"
Nov 03 20:39:48.547926 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.547893    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"]
Nov 03 20:39:48.549933 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.549918    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:48.596364 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.596339    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:48.596489 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.596375    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nqjx\" (UniqueName: \"kubernetes.io/projected/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-kube-api-access-5nqjx\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:48.596489 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.596400    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:48.596489 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.596449    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:48.596619 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.596489    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:48.596619 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.596518    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:48.652397 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.652374    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 03 20:39:48.652578 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.652443    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Nov 03 20:39:48.652578 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.652487    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Nov 03 20:39:48.652782 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.652767    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 03 20:39:48.653337 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.653321    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 03 20:39:48.693410 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.693387    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-glsfk\""
Nov 03 20:39:48.696713 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.696695    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:48.696792 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.696726    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhxc\" (UniqueName: \"kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc\") pod \"network-check-target-l9p4t\" (UID: \"6b982774-43c6-45ad-a3e3-77b7bc3d534e\") " pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:48.696792 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.696744    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nqjx\" (UniqueName: \"kubernetes.io/projected/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-kube-api-access-5nqjx\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:48.696792 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.696771    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:48.696936 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.696813    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:48.697498 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.697417    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:48.699204 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.699180    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:48.699280 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.699223    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:48.702251 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.702236    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 20:39:48.702332 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.702321    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 20:39:48.702899 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.702882    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"]
Nov 03 20:39:48.708725 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.708704    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:48.708856 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.708839    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:48.747811 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.747793    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 20:39:48.756570 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.756550    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nqjx\" (UniqueName: \"kubernetes.io/projected/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-kube-api-access-5nqjx\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:48.808324 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.808269    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 20:39:48.820082 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.820056    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lhxc\" (UniqueName: \"kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc\") pod \"network-check-target-l9p4t\" (UID: \"6b982774-43c6-45ad-a3e3-77b7bc3d534e\") " pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:48.861048 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:48.861019    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:49.016189 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:49.016160    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-mtgd7\""
Nov 03 20:39:49.019161 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:49.019142    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zvv9n\""
Nov 03 20:39:49.024243 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:49.024227    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:49.025798 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:49.025775    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j2mnh\""
Nov 03 20:39:49.028441 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:49.028424    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:49.034930 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:49.034911    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:49.227479 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:49.227449    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"]
Nov 03 20:39:49.230344 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:39:49.230315    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b5fff49_fcc5_419f_978f_afb7ea92e6c3.slice/crio-ea455e2d621c694b8e4c9b8c15b4a23909b85656c29641d9f26b7113965f5cae WatchSource:0}: Error finding container ea455e2d621c694b8e4c9b8c15b4a23909b85656c29641d9f26b7113965f5cae: Status 404 returned error can't find the container with id ea455e2d621c694b8e4c9b8c15b4a23909b85656c29641d9f26b7113965f5cae
Nov 03 20:39:49.400452 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:49.400388    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8jhcx"]
Nov 03 20:39:49.403274 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:39:49.403249    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9353a680_00bf_4d3a_8437_5a959a40a682.slice/crio-3c2c67ac7e6f40b18ce3cf9702b813f95c87ac950a1e17a2c3ee8347bf27bc2b WatchSource:0}: Error finding container 3c2c67ac7e6f40b18ce3cf9702b813f95c87ac950a1e17a2c3ee8347bf27bc2b: Status 404 returned error can't find the container with id 3c2c67ac7e6f40b18ce3cf9702b813f95c87ac950a1e17a2c3ee8347bf27bc2b
Nov 03 20:39:49.407638 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:49.407603    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l9p4t"]
Nov 03 20:39:49.410120 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:39:49.410101    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b982774_43c6_45ad_a3e3_77b7bc3d534e.slice/crio-ac0a7d762a7b28f0a3d5c76ce76eef22dd1f645edee406a56f21e1658de2918a WatchSource:0}: Error finding container ac0a7d762a7b28f0a3d5c76ce76eef22dd1f645edee406a56f21e1658de2918a: Status 404 returned error can't find the container with id ac0a7d762a7b28f0a3d5c76ce76eef22dd1f645edee406a56f21e1658de2918a
Nov 03 20:39:49.450407 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:49.450386    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hmnm7"]
Nov 03 20:39:49.452990 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:39:49.452971    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57913479_90b9_4db4_90f1_81cb81c2e5bb.slice/crio-36a041f139a3468d577f21f05184c14a1f54e077154cd6ee468329e3808ee152 WatchSource:0}: Error finding container 36a041f139a3468d577f21f05184c14a1f54e077154cd6ee468329e3808ee152: Status 404 returned error can't find the container with id 36a041f139a3468d577f21f05184c14a1f54e077154cd6ee468329e3808ee152
Nov 03 20:39:50.120503 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.120456    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hmnm7" event={"ID":"57913479-90b9-4db4-90f1-81cb81c2e5bb","Type":"ContainerStarted","Data":"36a041f139a3468d577f21f05184c14a1f54e077154cd6ee468329e3808ee152"}
Nov 03 20:39:50.121799 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.121768    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt" event={"ID":"1b5fff49-fcc5-419f-978f-afb7ea92e6c3","Type":"ContainerStarted","Data":"ea455e2d621c694b8e4c9b8c15b4a23909b85656c29641d9f26b7113965f5cae"}
Nov 03 20:39:50.122868 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.122846    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l9p4t" event={"ID":"6b982774-43c6-45ad-a3e3-77b7bc3d534e","Type":"ContainerStarted","Data":"ac0a7d762a7b28f0a3d5c76ce76eef22dd1f645edee406a56f21e1658de2918a"}
Nov 03 20:39:50.123885 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.123856    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8jhcx" event={"ID":"9353a680-00bf-4d3a-8437-5a959a40a682","Type":"ContainerStarted","Data":"3c2c67ac7e6f40b18ce3cf9702b813f95c87ac950a1e17a2c3ee8347bf27bc2b"}
Nov 03 20:39:50.259316 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.259282    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-hd6z4"]
Nov 03 20:39:50.261022 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.261007    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-hd6z4"
Nov 03 20:39:50.308444 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.308419    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42903eab-e4bc-4f97-9e73-1109e9f5e7ab-tmp\") pod \"downloads-6dd789c8b7-hd6z4\" (UID: \"42903eab-e4bc-4f97-9e73-1109e9f5e7ab\") " pod="openshift-console/downloads-6dd789c8b7-hd6z4"
Nov 03 20:39:50.308577 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.308454    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4jbc\" (UniqueName: \"kubernetes.io/projected/42903eab-e4bc-4f97-9e73-1109e9f5e7ab-kube-api-access-t4jbc\") pod \"downloads-6dd789c8b7-hd6z4\" (UID: \"42903eab-e4bc-4f97-9e73-1109e9f5e7ab\") " pod="openshift-console/downloads-6dd789c8b7-hd6z4"
Nov 03 20:39:50.313234 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.313208    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 03 20:39:50.313234 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.313233    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 03 20:39:50.362578 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.362554    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-l4h2k\""
Nov 03 20:39:50.365974 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.365942    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-hd6z4"]
Nov 03 20:39:50.409434 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.409365    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42903eab-e4bc-4f97-9e73-1109e9f5e7ab-tmp\") pod \"downloads-6dd789c8b7-hd6z4\" (UID: \"42903eab-e4bc-4f97-9e73-1109e9f5e7ab\") " pod="openshift-console/downloads-6dd789c8b7-hd6z4"
Nov 03 20:39:50.409434 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.409397    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4jbc\" (UniqueName: \"kubernetes.io/projected/42903eab-e4bc-4f97-9e73-1109e9f5e7ab-kube-api-access-t4jbc\") pod \"downloads-6dd789c8b7-hd6z4\" (UID: \"42903eab-e4bc-4f97-9e73-1109e9f5e7ab\") " pod="openshift-console/downloads-6dd789c8b7-hd6z4"
Nov 03 20:39:50.409849 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.409818    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42903eab-e4bc-4f97-9e73-1109e9f5e7ab-tmp\") pod \"downloads-6dd789c8b7-hd6z4\" (UID: \"42903eab-e4bc-4f97-9e73-1109e9f5e7ab\") " pod="openshift-console/downloads-6dd789c8b7-hd6z4"
Nov 03 20:39:50.472217 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.472187    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4jbc\" (UniqueName: \"kubernetes.io/projected/42903eab-e4bc-4f97-9e73-1109e9f5e7ab-kube-api-access-t4jbc\") pod \"downloads-6dd789c8b7-hd6z4\" (UID: \"42903eab-e4bc-4f97-9e73-1109e9f5e7ab\") " pod="openshift-console/downloads-6dd789c8b7-hd6z4"
Nov 03 20:39:50.571046 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:50.571012    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-hd6z4"
Nov 03 20:39:51.060120 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:51.060088    2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-hd6z4"]
Nov 03 20:39:51.108854 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:39:51.103336    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42903eab_e4bc_4f97_9e73_1109e9f5e7ab.slice/crio-ce777a6ba45a8a5134d45366ddf9787c04504e24c7b0575d8e061ee105c91d20 WatchSource:0}: Error finding container ce777a6ba45a8a5134d45366ddf9787c04504e24c7b0575d8e061ee105c91d20: Status 404 returned error can't find the container with id ce777a6ba45a8a5134d45366ddf9787c04504e24c7b0575d8e061ee105c91d20
Nov 03 20:39:51.130840 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:51.130806    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-hd6z4" event={"ID":"42903eab-e4bc-4f97-9e73-1109e9f5e7ab","Type":"ContainerStarted","Data":"ce777a6ba45a8a5134d45366ddf9787c04504e24c7b0575d8e061ee105c91d20"}
Nov 03 20:39:52.138185 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:52.138155    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hmnm7" event={"ID":"57913479-90b9-4db4-90f1-81cb81c2e5bb","Type":"ContainerStarted","Data":"ca56b6a8d664c088fc8679ccbe922ca6e8e85bc12057c0b64c207777ae33a6fd"}
Nov 03 20:39:52.138614 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:52.138214    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hmnm7" event={"ID":"57913479-90b9-4db4-90f1-81cb81c2e5bb","Type":"ContainerStarted","Data":"ed0273a1fc71947590b286ac575750cfaf2c48bb231c682f903825c4e498cefb"}
Nov 03 20:39:52.140930 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:52.140904    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt" event={"ID":"1b5fff49-fcc5-419f-978f-afb7ea92e6c3","Type":"ContainerStarted","Data":"b506958c3a060e3d3217a10bf5525f4ec327f04bd5c97c9cba1b76d58333baf1"}
Nov 03 20:39:52.141059 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:52.140955    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt" event={"ID":"1b5fff49-fcc5-419f-978f-afb7ea92e6c3","Type":"ContainerStarted","Data":"63c4328383265258547f34e09b0339df9adc1e236895cbcc94c9de8a9f600238"}
Nov 03 20:39:52.254803 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:52.254750    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hmnm7" podStartSLOduration=68.569869955 podStartE2EDuration="1m10.254730762s" totalImagesPullingTime="1.684860807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:38:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:52.253409581 +0000 UTC m=+69.988319478" watchObservedRunningTime="2026-11-03 20:39:52.254730762 +0000 UTC m=+69.989640653"
Nov 03 20:39:52.371724 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:52.371644    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt" podStartSLOduration=2.144468422 podStartE2EDuration="4.371592995s" totalImagesPullingTime="2.227124573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:39:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:52.371245894 +0000 UTC m=+70.106155773" watchObservedRunningTime="2026-11-03 20:39:52.371592995 +0000 UTC m=+70.106502874"
Nov 03 20:39:54.150120 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:54.150079    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l9p4t" event={"ID":"6b982774-43c6-45ad-a3e3-77b7bc3d534e","Type":"ContainerStarted","Data":"a3a7c4cf65ef9085b55c0186a701997515fd3131d3088ea8f1956d05a9fb0e7b"}
Nov 03 20:39:54.150521 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:54.150235    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:54.267424 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:54.267365    2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l9p4t" podStartSLOduration=67.223753999 podStartE2EDuration="1m11.267353785s" totalImagesPullingTime="4.043599786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:39:54.266480761 +0000 UTC m=+72.001390637" watchObservedRunningTime="2026-11-03 20:39:54.267353785 +0000 UTC m=+72.002263662"
Nov 03 20:39:55.105488 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.105450    2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:55.284909 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.284881    2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6sjbh"]
Nov 03 20:39:55.287280 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.287251    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.348795 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.348765    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-wtmp\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.348954 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.348811    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.348954 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.348839    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-root\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.348954 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.348903    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rbjq\" (UniqueName: \"kubernetes.io/projected/737d534b-bee4-44e9-ac5d-124b25de7f72-kube-api-access-4rbjq\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.349078 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.349008    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-textfile\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.349078 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.349043    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/737d534b-bee4-44e9-ac5d-124b25de7f72-metrics-client-ca\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.349172 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.349073    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.349172 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.349119    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-sys\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.349172 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.349144    2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-tls\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.350364 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.350329    2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 03 20:39:55.414076 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.413784    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 03 20:39:55.414076 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.413919    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rzwv7\""
Nov 03 20:39:55.414076 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.413988    2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 03 20:39:55.449456 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.449424    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-wtmp\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.449456 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.449461    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.449692 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.449486    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-root\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.449692 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.449506    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rbjq\" (UniqueName: \"kubernetes.io/projected/737d534b-bee4-44e9-ac5d-124b25de7f72-kube-api-access-4rbjq\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.449692 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.449531    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-textfile\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.449692 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.449557    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/737d534b-bee4-44e9-ac5d-124b25de7f72-metrics-client-ca\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.449692 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.449581    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.449692 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.449619    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-sys\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.449692 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.449640    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-wtmp\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.449692 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.449657    2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-tls\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.450033 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.450014    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-textfile\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.450090 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.450076    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-root\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.450611 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.450583    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.450741 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.450654    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-sys\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.451089 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.451060    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/737d534b-bee4-44e9-ac5d-124b25de7f72-metrics-client-ca\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.452784 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.452760    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-tls\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.467752 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.467724    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.511135 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.511102    2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rbjq\" (UniqueName: \"kubernetes.io/projected/737d534b-bee4-44e9-ac5d-124b25de7f72-kube-api-access-4rbjq\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.603687 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:55.603655    2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:55.615615 ip-10-0-142-223 kubenswrapper[2575]: W1103 20:39:55.615590    2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod737d534b_bee4_44e9_ac5d_124b25de7f72.slice/crio-23545138cff3fd148cdd66ddd42dc06a3ad812b695acbaf50f39a9db273b658f WatchSource:0}: Error finding container 23545138cff3fd148cdd66ddd42dc06a3ad812b695acbaf50f39a9db273b658f: Status 404 returned error can't find the container with id 23545138cff3fd148cdd66ddd42dc06a3ad812b695acbaf50f39a9db273b658f
Nov 03 20:39:56.158492 ip-10-0-142-223 kubenswrapper[2575]: I1103 20:39:56.158448    2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sjbh" event={"ID":"737d534b-bee4-44e9-ac5d-124b25de7f72","Type":"ContainerStarted","Data":"23545138cff3fd148cdd66ddd42dc06a3ad812b695acbaf50f39a9db273b658f"}
Nov 03 20:39:57.374039 ip-10-0-142-223 systemd[1]: Stopping Kubernetes Kubelet...
Nov 03 20:39:57.388161 ip-10-0-142-223 systemd[1]: kubelet.service: Deactivated successfully.
Nov 03 20:39:57.388342 ip-10-0-142-223 systemd[1]: Stopped Kubernetes Kubelet.
Nov 03 20:39:57.388394 ip-10-0-142-223 systemd[1]: kubelet.service: Consumed 3.640s CPU time.
Nov 03 20:39:57.405189 ip-10-0-142-223 systemd[1]: Starting Kubernetes Kubelet...
Nov 03 20:39:57.489448 ip-10-0-142-223 kubenswrapper[4993]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 20:39:57.489448 ip-10-0-142-223 kubenswrapper[4993]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 03 20:39:57.489448 ip-10-0-142-223 kubenswrapper[4993]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 20:39:57.489448 ip-10-0-142-223 kubenswrapper[4993]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 03 20:39:57.491518 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491430    4993 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:57.491518 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491518    4993 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491522    4993 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491525    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491528    4993 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491531    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491534    4993 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491538    4993 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491543    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491546    4993 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491549    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491553    4993 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491555    4993 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491558    4993 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491560    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491563    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491566    4993 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491568    4993 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491570    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491573    4993 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:57.491586 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491576    4993 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491579    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491581    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491584    4993 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491586    4993 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491589    4993 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491592    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491601    4993 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491604    4993 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491606    4993 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491609    4993 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491611    4993 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491614    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491617    4993 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491619    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491622    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491641    4993 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491645    4993 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491647    4993 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491651    4993 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:57.492079 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491655    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491658    4993 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491661    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491664    4993 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491668    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491670    4993 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491673    4993 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491676    4993 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491679    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491681    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491684    4993 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491687    4993 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491690    4993 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491693    4993 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491697    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491700    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491703    4993 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491706    4993 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491709    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:57.492562 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491711    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491715    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491718    4993 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491721    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491723    4993 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491726    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491729    4993 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491731    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491734    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491736    4993 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491739    4993 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491741    4993 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491744    4993 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491746    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491749    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491751    4993 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491754    4993 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491757    4993 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491759    4993 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491762    4993 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:57.493088 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491764    4993 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491767    4993 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491770    4993 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491773    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491776    4993 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491778    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491780    4993 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491784    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491786    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491789    4993 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491792    4993 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491794    4993 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491797    4993 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491799    4993 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491802    4993 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491806    4993 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491809    4993 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491811    4993 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491814    4993 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491816    4993 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:57.493568 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491819    4993 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491821    4993 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491823    4993 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491826    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491828    4993 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491831    4993 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.491834    4993 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492588    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492596    4993 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492599    4993 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492602    4993 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492605    4993 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492608    4993 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492610    4993 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492614    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492617    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492619    4993 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492622    4993 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492644    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:57.494113 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492649    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492653    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492657    4993 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492661    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492665    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492670    4993 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492673    4993 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492675    4993 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492686    4993 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492688    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492691    4993 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492694    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492697    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492699    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492702    4993 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492704    4993 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492707    4993 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492709    4993 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492712    4993 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492715    4993 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:57.494589 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492717    4993 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492720    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492722    4993 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492725    4993 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492727    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492730    4993 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492733    4993 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492735    4993 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492738    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492741    4993 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492743    4993 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492746    4993 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492749    4993 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492751    4993 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492754    4993 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492757    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492762    4993 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492766    4993 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492770    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492772    4993 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:57.495146 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492775    4993 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492778    4993 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492781    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492783    4993 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492786    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492788    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492791    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492794    4993 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492796    4993 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492799    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492801    4993 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492804    4993 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492806    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492809    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492811    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492814    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492816    4993 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492819    4993 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492821    4993 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:57.495643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492824    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492827    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492829    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492832    4993 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492834    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492839    4993 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492843    4993 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492846    4993 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492848    4993 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492851    4993 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492854    4993 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492856    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492860    4993 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492862    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492865    4993 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492868    4993 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492870    4993 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492873    4993 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492875    4993 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492878    4993 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:57.496114 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492880    4993 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492883    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492885    4993 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492888    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492892    4993 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492895    4993 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492898    4993 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492900    4993 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492903    4993 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492905    4993 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492907    4993 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492910    4993 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492913    4993 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492915    4993 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.492918    4993 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493417    4993 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493422    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493425    4993 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493428    4993 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493431    4993 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:57.496597 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493433    4993 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493436    4993 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493439    4993 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493441    4993 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493444    4993 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493446    4993 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493449    4993 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493452    4993 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493455    4993 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493457    4993 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493460    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493463    4993 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493465    4993 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493468    4993 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493471    4993 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493474    4993 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493479    4993 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493482    4993 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493485    4993 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493487    4993 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:57.497101 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493490    4993 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493493    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493495    4993 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493498    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493501    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493503    4993 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493506    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493509    4993 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493511    4993 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493514    4993 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493516    4993 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493519    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493521    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493524    4993 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493526    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493529    4993 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493531    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493534    4993 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493536    4993 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:57.497643 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493539    4993 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493542    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493544    4993 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493546    4993 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493549    4993 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493551    4993 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493554    4993 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493556    4993 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493559    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493563    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493565    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493568    4993 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493571    4993 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493575    4993 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493578    4993 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493581    4993 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493583    4993 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493586    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493588    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493591    4993 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:57.498092 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493593    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493596    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493598    4993 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493602    4993 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493605    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493608    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493611    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493613    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493616    4993 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493618    4993 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493644    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493648    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493650    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493653    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493656    4993 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493659    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493661    4993 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493664    4993 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493667    4993 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:57.498585 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493669    4993 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493672    4993 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493675    4993 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493678    4993 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493681    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493683    4993 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493686    4993 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493688    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493691    4993 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493693    4993 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493696    4993 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493698    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493702    4993 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493706    4993 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493709    4993 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493712    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493715    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493717    4993 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493720    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:57.499363 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493722    4993 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493725    4993 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493728    4993 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.493731    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493826    4993 flags.go:64] FLAG: --address="0.0.0.0"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493834    4993 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493840    4993 flags.go:64] FLAG: --anonymous-auth="true"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493845    4993 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493850    4993 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493854    4993 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493858    4993 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493863    4993 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493866    4993 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493869    4993 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493872    4993 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493876    4993 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493879    4993 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493882    4993 flags.go:64] FLAG: --cgroup-root=""
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493885    4993 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493888    4993 flags.go:64] FLAG: --client-ca-file=""
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493891    4993 flags.go:64] FLAG: --cloud-provider="external"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493894    4993 flags.go:64] FLAG: --cluster-dns="[]"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493897    4993 flags.go:64] FLAG: --cluster-domain=""
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493900    4993 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 03 20:39:57.500044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493903    4993 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493906    4993 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493909    4993 flags.go:64] FLAG: --container-log-max-files="5"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493913    4993 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493916    4993 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493919    4993 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493922    4993 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493925    4993 flags.go:64] FLAG: --contention-profiling="false"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493928    4993 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493930    4993 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493933    4993 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493937    4993 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493943    4993 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493946    4993 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493949    4993 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493952    4993 flags.go:64] FLAG: --enable-load-reader="false"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493955    4993 flags.go:64] FLAG: --enable-server="true"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493957    4993 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493962    4993 flags.go:64] FLAG: --event-burst="100"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493966    4993 flags.go:64] FLAG: --event-qps="50"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493969    4993 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493972    4993 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493975    4993 flags.go:64] FLAG: --eviction-hard=""
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493979    4993 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 03 20:39:57.500643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493982    4993 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493985    4993 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493988    4993 flags.go:64] FLAG: --eviction-soft=""
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493991    4993 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493994    4993 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.493997    4993 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494000    4993 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494003    4993 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494006    4993 flags.go:64] FLAG: --fail-swap-on="true"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494008    4993 flags.go:64] FLAG: --feature-gates=""
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494012    4993 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494015    4993 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494018    4993 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494021    4993 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494024    4993 flags.go:64] FLAG: --healthz-port="10248"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494028    4993 flags.go:64] FLAG: --help="false"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494031    4993 flags.go:64] FLAG: --hostname-override="ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494034    4993 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494037    4993 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494040    4993 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494044    4993 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494048    4993 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494051    4993 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 03 20:39:57.501231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494054    4993 flags.go:64] FLAG: --image-service-endpoint=""
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494056    4993 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494059    4993 flags.go:64] FLAG: --kube-api-burst="100"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494062    4993 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494066    4993 flags.go:64] FLAG: --kube-api-qps="50"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494069    4993 flags.go:64] FLAG: --kube-reserved=""
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494072    4993 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494074    4993 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494077    4993 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494080    4993 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494083    4993 flags.go:64] FLAG: --lock-file=""
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494086    4993 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494089    4993 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494092    4993 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494097    4993 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494100    4993 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494103    4993 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494106    4993 flags.go:64] FLAG: --logging-format="text"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494109    4993 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494113    4993 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494116    4993 flags.go:64] FLAG: --manifest-url=""
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494119    4993 flags.go:64] FLAG: --manifest-url-header=""
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494123    4993 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494126    4993 flags.go:64] FLAG: --max-open-files="1000000"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494130    4993 flags.go:64] FLAG: --max-pods="110"
Nov 03 20:39:57.501846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494135    4993 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494138    4993 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494141    4993 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494144    4993 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494149    4993 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494152    4993 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494155    4993 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494162    4993 flags.go:64] FLAG: --node-status-max-images="50"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494165    4993 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494168    4993 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494171    4993 flags.go:64] FLAG: --pod-cidr=""
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494173    4993 flags.go:64] FLAG: --pod-manifest-path=""
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494177    4993 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494180    4993 flags.go:64] FLAG: --pods-per-core="0"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494183    4993 flags.go:64] FLAG: --port="10250"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494186    4993 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494189    4993 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-062bd8430de494aef"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494192    4993 flags.go:64] FLAG: --qos-reserved=""
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494195    4993 flags.go:64] FLAG: --read-only-port="10255"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494198    4993 flags.go:64] FLAG: --register-node="true"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494201    4993 flags.go:64] FLAG: --register-with-taints=""
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494204    4993 flags.go:64] FLAG: --registry-burst="10"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494207    4993 flags.go:64] FLAG: --registry-qps="5"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494210    4993 flags.go:64] FLAG: --reserved-cpus=""
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494213    4993 flags.go:64] FLAG: --reserved-memory=""
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494216    4993 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 03 20:39:57.502470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494219    4993 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494222    4993 flags.go:64] FLAG: --rotate-certificates="false"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494225    4993 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494228    4993 flags.go:64] FLAG: --runonce="false"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494231    4993 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494234    4993 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494238    4993 flags.go:64] FLAG: --seccomp-default="false"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494241    4993 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494243    4993 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494247    4993 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494251    4993 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494254    4993 flags.go:64] FLAG: --storage-driver-password="root"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494257    4993 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494260    4993 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494263    4993 flags.go:64] FLAG: --storage-driver-user="root"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494266    4993 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494269    4993 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494271    4993 flags.go:64] FLAG: --system-cgroups=""
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494274    4993 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494280    4993 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494283    4993 flags.go:64] FLAG: --tls-cert-file=""
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494286    4993 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494290    4993 flags.go:64] FLAG: --tls-min-version=""
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494293    4993 flags.go:64] FLAG: --tls-private-key-file=""
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494295    4993 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 03 20:39:57.503132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494299    4993 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494302    4993 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494305    4993 flags.go:64] FLAG: --v="2"
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494309    4993 flags.go:64] FLAG: --version="false"
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494313    4993 flags.go:64] FLAG: --vmodule=""
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494317    4993 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494320    4993 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494455    4993 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494461    4993 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494465    4993 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494468    4993 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494471    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494474    4993 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494478    4993 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494481    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494483    4993 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494486    4993 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494490    4993 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494492    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494495    4993 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494498    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:57.503759 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494500    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494503    4993 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494505    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494508    4993 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494510    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494513    4993 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494516    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494519    4993 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494521    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494524    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494527    4993 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494530    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494532    4993 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494535    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494537    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494540    4993 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494542    4993 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494545    4993 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494547    4993 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:57.504328 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494551    4993 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494555    4993 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494557    4993 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494560    4993 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494563    4993 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494566    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494569    4993 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494571    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494574    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494577    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494580    4993 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494582    4993 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494585    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494587    4993 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494590    4993 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494592    4993 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494595    4993 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494598    4993 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494600    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494603    4993 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:57.505050 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494607    4993 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494609    4993 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494612    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494614    4993 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494617    4993 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494619    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494622    4993 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494638    4993 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494641    4993 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494644    4993 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494646    4993 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494649    4993 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494652    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494654    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494657    4993 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494660    4993 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494662    4993 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494667    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494670    4993 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494673    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:57.505822 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494675    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494679    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494682    4993 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494685    4993 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494687    4993 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494690    4993 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494693    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494696    4993 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494698    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494701    4993 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494704    4993 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494706    4993 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494709    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494712    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494714    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494717    4993 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494719    4993 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494721    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494724    4993 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494727    4993 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:57.506787 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494729    4993 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494732    4993 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494735    4993 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494737    4993 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494764    4993 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494768    4993 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494770    4993 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494773    4993 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494775    4993 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494778    4993 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494782    4993 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494784    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494787    4993 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.494817    4993 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494851    4993 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494855    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:57.507355 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494858    4993 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494861    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494864    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494867    4993 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494870    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494873    4993 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494875    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494878    4993 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494881    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494884    4993 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494887    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494890    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494892    4993 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494895    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494898    4993 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494900    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494903    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494905    4993 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494908    4993 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:57.507857 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494911    4993 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494913    4993 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494916    4993 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494920    4993 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494922    4993 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494925    4993 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494928    4993 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494931    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494934    4993 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494937    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494939    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494943    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494946    4993 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494948    4993 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494951    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494954    4993 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494956    4993 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494959    4993 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494961    4993 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:57.508444 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494964    4993 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494966    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494969    4993 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494971    4993 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494974    4993 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494977    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494979    4993 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494982    4993 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494984    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494987    4993 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494989    4993 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494992    4993 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494994    4993 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494997    4993 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.494999    4993 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495002    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495004    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495007    4993 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495009    4993 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:57.508920 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495012    4993 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495016    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495019    4993 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495022    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495025    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495028    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495031    4993 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495033    4993 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495036    4993 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495039    4993 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495041    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495044    4993 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495046    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495049    4993 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495052    4993 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495054    4993 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495057    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495059    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495062    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495065    4993 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:57.509392 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495067    4993 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495070    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495072    4993 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495075    4993 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495077    4993 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495080    4993 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495082    4993 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495085    4993 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495087    4993 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495090    4993 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495093    4993 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495095    4993 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495098    4993 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495101    4993 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495104    4993 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495106    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495109    4993 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495112    4993 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495116    4993 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495119    4993 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:57.509963 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495123    4993 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495126    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495128    4993 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495131    4993 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495133    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495136    4993 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.495138    4993 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.499336    4993 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.499388    4993 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.499416    4993 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.499424    4993 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499818    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499832    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499836    4993 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499839    4993 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499843    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499846    4993 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499850    4993 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499853    4993 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:57.510436 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499856    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499858    4993 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499861    4993 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499864    4993 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499867    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499870    4993 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499873    4993 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499875    4993 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499878    4993 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499881    4993 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499886    4993 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499891    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499895    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499898    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499901    4993 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499904    4993 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499907    4993 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499909    4993 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499912    4993 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:57.510916 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499915    4993 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499918    4993 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499921    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499924    4993 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499927    4993 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499930    4993 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499933    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499936    4993 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499938    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499941    4993 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499943    4993 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499946    4993 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499948    4993 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499951    4993 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499954    4993 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499956    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499959    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499961    4993 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499964    4993 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:57.511368 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499966    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499969    4993 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499972    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499974    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499977    4993 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499979    4993 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499982    4993 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499984    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499987    4993 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499990    4993 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499992    4993 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499995    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.499999    4993 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500002    4993 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500004    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500007    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500010    4993 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500013    4993 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500016    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:57.511850 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500018    4993 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500021    4993 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500023    4993 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500026    4993 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500028    4993 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500031    4993 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500034    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500036    4993 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500038    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500041    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500043    4993 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500046    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500048    4993 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500051    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500055    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500058    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500060    4993 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500063    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500066    4993 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500068    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:57.512302 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500070    4993 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500073    4993 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500075    4993 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500078    4993 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500081    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500083    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500086    4993 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500088    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500091    4993 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500094    4993 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500096    4993 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500099    4993 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500101    4993 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500104    4993 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500106    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500109    4993 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500111    4993 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500114    4993 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500116    4993 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500118    4993 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:57.512805 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500121    4993 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.500150    4993 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500288    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500294    4993 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500297    4993 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500300    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500303    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500307    4993 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500310    4993 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500313    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500315    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500318    4993 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500320    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500323    4993 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500325    4993 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500327    4993 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:57.513279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500330    4993 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500333    4993 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500335    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500337    4993 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500340    4993 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500343    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500345    4993 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500348    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500350    4993 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500352    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500355    4993 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500358    4993 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500360    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500362    4993 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500365    4993 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500367    4993 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500370    4993 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500372    4993 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500374    4993 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500377    4993 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:57.513721 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500379    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500382    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500384    4993 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500387    4993 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500390    4993 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500393    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500395    4993 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500398    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500400    4993 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500402    4993 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500405    4993 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500407    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500410    4993 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500413    4993 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500415    4993 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500417    4993 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500420    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500422    4993 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500425    4993 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500427    4993 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:57.514242 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500430    4993 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500432    4993 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500435    4993 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500437    4993 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500439    4993 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500442    4993 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500445    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500447    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500450    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500452    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500455    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500457    4993 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500460    4993 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500462    4993 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500464    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500467    4993 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500469    4993 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500472    4993 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500475    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:57.514798 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500478    4993 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500481    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500484    4993 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500486    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500490    4993 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500494    4993 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500497    4993 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500500    4993 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500503    4993 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500505    4993 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500508    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500511    4993 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500513    4993 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500516    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500518    4993 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500521    4993 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500523    4993 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500526    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500528    4993 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:57.515465 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500535    4993 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500537    4993 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500540    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500542    4993 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500545    4993 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500547    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500550    4993 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500552    4993 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500556    4993 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500560    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500562    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500565    4993 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500569    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500572    4993 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.500595    4993 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500646    4993 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 03 20:39:57.516224 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500653    4993 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500657    4993 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500660    4993 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500663    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500666    4993 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500669    4993 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500671    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500674    4993 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500676    4993 feature_gate.go:386] unrecognized feature gate: Example2
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500679    4993 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500681    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500684    4993 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500687    4993 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500689    4993 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500692    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500694    4993 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500697    4993 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500701    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500704    4993 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500706    4993 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 03 20:39:57.516785 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500710    4993 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500714    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500717    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500719    4993 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500722    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500725    4993 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500729    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500732    4993 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500734    4993 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500738    4993 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500740    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500743    4993 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500746    4993 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500748    4993 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500751    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500753    4993 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500755    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500758    4993 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500760    4993 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 03 20:39:57.517279 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500763    4993 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500765    4993 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500768    4993 feature_gate.go:386] unrecognized feature gate: Example
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500770    4993 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500773    4993 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500775    4993 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500778    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500780    4993 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500783    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500785    4993 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500789    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500792    4993 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500795    4993 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500797    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500800    4993 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500802    4993 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500805    4993 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500807    4993 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500809    4993 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 03 20:39:57.517893 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500813    4993 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500816    4993 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500818    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500821    4993 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500824    4993 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500827    4993 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500829    4993 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500832    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500835    4993 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500837    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500840    4993 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500842    4993 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500844    4993 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500847    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500849    4993 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500852    4993 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500854    4993 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500857    4993 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500859    4993 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500862    4993 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 03 20:39:57.518438 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500865    4993 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500867    4993 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500870    4993 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500874    4993 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500876    4993 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500879    4993 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500881    4993 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500885    4993 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500887    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500890    4993 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500892    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500895    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500899    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500902    4993 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500905    4993 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500907    4993 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500910    4993 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500913    4993 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500916    4993 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 03 20:39:57.519021 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500918    4993 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 03 20:39:57.519515 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500921    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 03 20:39:57.519515 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500924    4993 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 03 20:39:57.519515 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500927    4993 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 03 20:39:57.519515 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500930    4993 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 03 20:39:57.519515 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500932    4993 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 03 20:39:57.519515 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500937    4993 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 03 20:39:57.519515 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:57.500940    4993 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 03 20:39:57.519515 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.500994    4993 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 03 20:39:57.519515 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.501916    4993 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 03 20:39:57.519515 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.501972    4993 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 03 20:39:57.519515 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.502136    4993 server.go:1014] "Starting client certificate rotation"
Nov 03 20:39:57.519515 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.502277    4993 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 03 20:39:57.519515 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.502318    4993 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-02 20:33:42 +0000 UTC" deadline="2028-07-12 03:09:42.96050505 +0000 UTC"
Nov 03 20:39:57.519515 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.502346    4993 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14790h29m45.458161131s"
Nov 03 20:39:57.519515 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.502572    4993 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 20:39:57.519969 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.503651    4993 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 03 20:39:57.519969 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.504873    4993 log.go:25] "Validated CRI v1 runtime API"
Nov 03 20:39:57.519969 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.506955    4993 log.go:25] "Validated CRI v1 image API"
Nov 03 20:39:57.519969 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.507598    4993 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 03 20:39:57.519969 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.509288    4993 fs.go:135] Filesystem UUIDs: map[4e648b96-e44a-4406-8298-7e74fbc6b36c:/dev/nvme0n1p4 5ddd11bb-eac6-4f1b-8221-a69a3211d16f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Nov 03 20:39:57.520120 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.509308    4993 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/04aa687fe51e5801032b5505b2ed2d459b20773723b08d7fefc0815ce364e082/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/04aa687fe51e5801032b5505b2ed2d459b20773723b08d7fefc0815ce364e082/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/124bfbdf1e4804f9ba9772a37c4c7a844ecefce618897fbf97094e9f69883a9b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/124bfbdf1e4804f9ba9772a37c4c7a844ecefce618897fbf97094e9f69883a9b/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/23545138cff3fd148cdd66ddd42dc06a3ad812b695acbaf50f39a9db273b658f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/23545138cff3fd148cdd66ddd42dc06a3ad812b695acbaf50f39a9db273b658f/userdata/shm major:0 minor:319 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/264d7531f8500f215d53e74449911b84b203f7bbeb4549e3f9ab721c1d5ce85f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/264d7531f8500f215d53e74449911b84b203f7bbeb4549e3f9ab721c1d5ce85f/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2da42e99af23864aab781534a38b114b3161276f81da308f8a18052b84a7b741/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2da42e99af23864aab781534a38b114b3161276f81da308f8a18052b84a7b741/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/36a041f139a3468d577f21f05184c14a1f54e077154cd6ee468329e3808ee152/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/36a041f139a3468d577f21f05184c14a1f54e077154cd6ee468329e3808ee152/userdata/shm major:0 minor:273 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3c2c67ac7e6f40b18ce3cf9702b813f95c87ac950a1e17a2c3ee8347bf27bc2b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3c2c67ac7e6f40b18ce3cf9702b813f95c87ac950a1e17a2c3ee8347bf27bc2b/userdata/shm major:0 minor:269 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/579eab1e7331b2200a1eb348da970ca5a5300f140c69684d3c64ce632b879e61/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/579eab1e7331b2200a1eb348da970ca5a5300f140c69684d3c64ce632b879e61/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/596edf3c444e2d280e16329d46079439a81596ec44f3107fc1442753f79e2b8d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/596edf3c444e2d280e16329d46079439a81596ec44f3107fc1442753f79e2b8d/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/62995e99c79deae78ab49a3f98e520833e6ff056e23073fe3b09ddfc5f8a8e5d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/62995e99c79deae78ab49a3f98e520833e6ff056e23073fe3b09ddfc5f8a8e5d/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a168d8aade5dd2c4b5cfcf08cbb196fb2e610039d92d4de120ea4f66393a6a03/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a168d8aade5dd2c4b5cfcf08cbb196fb2e610039d92d4de120ea4f66393a6a03/userdata/shm major:0 minor:229 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a24e27a43277303c542a3fab007c981392e4c6cb4eca6bb9dbef0a265fbdc3c5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a24e27a43277303c542a3fab007c981392e4c6cb4eca6bb9dbef0a265fbdc3c5/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ac0a7d762a7b28f0a3d5c76ce76eef22dd1f645edee406a56f21e1658de2918a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ac0a7d762a7b28f0a3d5c76ce76eef22dd1f645edee406a56f21e1658de2918a/userdata/shm major:0 minor:271 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ce777a6ba45a8a5134d45366ddf9787c04504e24c7b0575d8e061ee105c91d20/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ce777a6ba45a8a5134d45366ddf9787c04504e24c7b0575d8e061ee105c91d20/userdata/shm major:0 minor:284 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d27d19c6586f35ebee131ffe2dc27cd671d13753bdf7c0c7e950c8f166acbf38/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d27d19c6586f35ebee131ffe2dc27cd671d13753bdf7c0c7e950c8f166acbf38/userdata/shm major:0 minor:79 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d721f687bcff21f43e7a90eb4e2129c0f46ee36289c481e438e4e3cabbdc9ff3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d721f687bcff21f43e7a90eb4e2129c0f46ee36289c481e438e4e3cabbdc9ff3/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e00279b084939ae0376b5e253923ddd2b2b67c4dd139b86e80849df2438b28d5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e00279b084939ae0376b5e253923ddd2b2b67c4dd139b86e80849df2438b28d5/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e88f6a237b09b306079b4a9f5aa3eeca9a6fe5b518e5c10d63ddb151a39c9d3d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e88f6a237b09b306079b4a9f5aa3eeca9a6fe5b518e5c10d63ddb151a39c9d3d/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e92ecfcfeb535a68e348a0311cf7def199e9eb25188161b246dfbfc28236c9d5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e92ecfcfeb535a68e348a0311cf7def199e9eb25188161b246dfbfc28236c9d5/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ea455e2d621c694b8e4c9b8c15b4a23909b85656c29641d9f26b7113965f5cae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ea455e2d621c694b8e4c9b8c15b4a23909b85656c29641d9f26b7113965f5cae/userdata/shm major:0 minor:267 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0ffcf342-bccb-4362-b79e-8faf3d337b17/volumes/kubernetes.io~projected/kube-api-access-t5vgd:{mountpoint:/var/lib/kubelet/pods/0ffcf342-bccb-4362-b79e-8faf3d337b17/volumes/kubernetes.io~projected/kube-api-access-t5vgd major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/10a65b13-a9e4-4355-8bac-fab245388f16/volumes/kubernetes.io~projected/kube-api-access-kbpzw:{mountpoint:/var/lib/kubelet/pods/10a65b13-a9e4-4355-8bac-fab245388f16/volumes/kubernetes.io~projected/kube-api-access-kbpzw major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11516670-b4cd-43cc-8c41-64e5bfed2b5a/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/11516670-b4cd-43cc-8c41-64e5bfed2b5a/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11516670-b4cd-43cc-8c41-64e5bfed2b5a/volumes/kubernetes.io~projected/kube-api-access-9bffp:{mountpoint:/var/lib/kubelet/pods/11516670-b4cd-43cc-8c41-64e5bfed2b5a/volumes/kubernetes.io~projected/kube-api-access-9bffp major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11516670-b4cd-43cc-8c41-64e5bfed2b5a/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/11516670-b4cd-43cc-8c41-64e5bfed2b5a/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1169f5bf-4ec8-4f85-aba9-857c3a666a3f/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/1169f5bf-4ec8-4f85-aba9-857c3a666a3f/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1169f5bf-4ec8-4f85-aba9-857c3a666a3f/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/1169f5bf-4ec8-4f85-aba9-857c3a666a3f/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1169f5bf-4ec8-4f85-aba9-857c3a666a3f/volumes/kubernetes.io~projected/kube-api-access-sl9lr:{mountpoint:/var/lib/kubelet/pods/1169f5bf-4ec8-4f85-aba9-857c3a666a3f/volumes/kubernetes.io~projected/kube-api-access-sl9lr major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/131affdb-a6b8-457d-84b6-ffceca9f2bfd/volumes/kubernetes.io~projected/kube-api-access-hkzsl:{mountpoint:/var/lib/kubelet/pods/131affdb-a6b8-457d-84b6-ffceca9f2bfd/volumes/kubernetes.io~projected/kube-api-access-hkzsl major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/131affdb-a6b8-457d-84b6-ffceca9f2bfd/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/131affdb-a6b8-457d-84b6-ffceca9f2bfd/volumes/kubernetes.io~secret/cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1b5fff49-fcc5-419f-978f-afb7ea92e6c3/volumes/kubernetes.io~projected/kube-api-access-5nqjx:{mountpoint:/var/lib/kubelet/pods/1b5fff49-fcc5-419f-978f-afb7ea92e6c3/volumes/kubernetes.io~projected/kube-api-access-5nqjx major:0 minor:265 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1b5fff49-fcc5-419f-978f-afb7ea92e6c3/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/1b5fff49-fcc5-419f-978f-afb7ea92e6c3/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:257 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1b5fff49-fcc5-419f-978f-afb7ea92e6c3/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/1b5fff49-fcc5-419f-978f-afb7ea92e6c3/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:262 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/32cf1fce-f61e-4870-aff9-a9171e90b09f/volumes/kubernetes.io~projected/kube-api-access-wmdh2:{mountpoint:/var/lib/kubelet/pods/32cf1fce-f61e-4870-aff9-a9171e90b09f/volumes/kubernetes.io~projected/kube-api-access-wmdh2 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/346aebd9-354c-4683-9eff-58058e9919c7/volumes/kubernetes.io~projected/kube-api-access-2f6kw:{mountpoint:/var/lib/kubelet/pods/346aebd9-354c-4683-9eff-58058e9919c7/volumes/kubernetes.io~projected/kube-api-access-2f6kw major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/346aebd9-354c-4683-9eff-58058e9919c7/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/346aebd9-354c-4683-9eff-58058e9919c7/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42903eab-e4bc-4f97-9e73-1109e9f5e7ab/volumes/kubernetes.io~projected/kube-api-access-t4jbc:{mountpoint:/var/lib/kubelet/pods/42903eab-e4bc-4f97-9e73-1109e9f5e7ab/volumes/kubernetes.io~projected/kube-api-access-t4jbc major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2/volumes/kubernetes.io~projected/kube-api-access-nrcsl:{mountpoint:/var/lib/kubelet/pods/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2/volumes/kubernetes.io~projected/kube-api-access-nrcsl major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/57913479-90b9-4db4-90f1-81cb81c2e5bb/volumes/kubernetes.io~projected/kube-api-access-87s22:{mountpoint:/var/lib/kubelet/pods/57913479-90b9-4db4-90f1-81cb81c2e5bb/volumes/kubernetes.io~projected/kube-api-access-87s22 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/57913479-90b9-4db4-90f1-81cb81c2e5bb/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/57913479-90b9-4db4-90f1-81cb81c2e5bb/volumes/kubernetes.io~secret/metrics-certs major:0 minor:264 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6b982774-43c6-45ad-a3e3-77b7bc3d534e/volumes/kubernetes.io~projected/kube-api-access-5lhxc:{mountpoint:/var/lib/kubelet/pods/6b982774-43c6-45ad-a3e3-77b7bc3d534e/volumes/kubernetes.io~projected/kube-api-access-5lhxc major:0 minor:266 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/737d534b-bee4-44e9-ac5d-124b25de7f72/volumes/kubernetes.io~projected/kube-api-access-4rbjq:{mountpoint:/var/lib/kubelet/pods/737d534b-bee4-44e9-ac5d-124b25de7f72/volumes/kubernetes.io~projected/kube-api-access-4rbjq major:0 minor:318 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/737d534b-bee4-44e9-ac5d-124b25de7f72/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/737d534b-bee4-44e9-ac5d-124b25de7f72/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:317 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/737d534b-bee4-44e9-ac5d-124b25de7f72/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/737d534b-bee4-44e9-ac5d-124b25de7f72/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:312 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43/volumes/kubernetes.io~projected/kube-api-access-qlfjt:{mountpoint:/var/lib/kubelet/pods/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43/volumes/kubernetes.io~projected/kube-api-access-qlfjt major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9353a680-00bf-4d3a-8437-5a959a40a682/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/9353a680-00bf-4d3a-8437-5a959a40a682/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:263 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b6fe25b0-03dc-434d-957b-5f25c8d5c141/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/b6fe25b0-03dc-434d-957b-5f25c8d5c141/volumes/kubernetes.io~secret/agent-certs major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c4cacc5f-ade7-47d9-af40-78cdc6e090ba/volumes/kubernetes.io~secret/tls-certificates:{mountpoint:/var/lib/kubelet/pods/c4cacc5f-ade7-47d9-af40-78cdc6e090ba/volumes/kubernetes.io~secret/tls-certificates major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e5e0251d-3ac4-4b42-9175-39ffd668896f/volumes/kubernetes.io~projected/kube-api-access-j726r:{mountpoint:/var/lib/kubelet/pods/e5e0251d-3ac4-4b42-9175-39ffd668896f/volumes/kubernetes.io~projected/kube-api-access-j726r major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e658bf30-634f-4db1-9b41-280a746540a8/volumes/kubernetes.io~projected/kube-api-access-dbw7c:{mountpoint:/var/lib/kubelet/pods/e658bf30-634f-4db1-9b41-280a746540a8/volumes/kubernetes.io~projected/kube-api-access-dbw7c major:0 minor:64 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/fbe780c3d3db265b422d41977fc1e87837b1829c48d086ac7fae850ee89ac383/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/c8d97b97fd52807c3358494f72e7b44896a78507e63ab58772c70df9611bfd41/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/ff576a96c9b3f69dc9de23bb75fbacc36c0aa41aa786721833661817aa59ac57/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/22cb6cfd5fc86e4f547b4b25bf33e0d165d7cf0ee70c007d36539c4edfff294e/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/8e484ec9502df9c36eec9c82509f00e19a90020cf6fb6b892d1c62dcc0cd200a/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/f251e0172f5a67e4ae1381d0e02d817e29020bdb1ffc6892fb3f571b4ee72131/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/7e9765f477e89b6476bdca857cfa537c55a58aa8c7779bcc86725d33bd0d752f/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/bbab73e9ebdb058e714a724238ab0beff9b2fd272b622c2f90963106148363f5/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/53bbeee72ee28a3c0ac98fb176607b1ce03d4ae26639e8c924f9219eac984948/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/4c6af34b154e851295aa09f03a2f8ba8861b8eef728eff4418970a8bff7da79a/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-141:{mountpoint:/var/lib/containers/storage/overlay/d5852b5bab410a7da5810cba148a312e496803a19ad1d14c1e3eb04f3efd5a19/merged major:0 minor:141 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/d51071407a5efb3a4dea0bd4b3f7d396d0e0188c2b4d1dfc654a012c11445f98/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/e528256f4205fde1657ed122d68d36e159c2ddb65c8226b8d8a39092d19619b7/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/5bcf720efbdd39c1643d2d5bed15863ece659bf4ca8e2420cca7be0b5f436fd6/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/faecb97efbe590f9702d0cf55ee742350e74ae7abf599746a144a6dd0185432d/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/24a88d54ac7f77146e2c632cbbb09ff8b8bc8ecec1338632f5394fd7a6d518d1/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/37ded39c69fd5323afba8d632dd515d7944802cd9333bfceadce8b5f5abefec3/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/f0fcbda2c40bc7df6090ba82e4083421e3842d3e527220da3ce7d3e18b8bd75e/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/a8a6ef892e3da6f776f17620fefe8c8308f7a66cf79170c58bc4f87b5ef7873f/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/54e8ec21cc0d07e6e57340e03a3b148504bbb1615fecec9071fa8970d975f0f0/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/9ae8f026aa121c9d62f2382f4491a7f99f539cc6c316849524eb9228f2ab7cac/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/c6cac12ed837c85b42a64ad9ecefbb74df4d2701871a6cde1deb816e931cbb32/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/eb4e7fc0e93207a8d1320de976cd088c3b11d3acf0ede3ee88c0c729cf214d1c/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/6c6a6ff6bb92c2ebb5674d2c50cf401c35f46aa30d4677497680a41f667a9f0f/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-220:{mountpoint:/var/lib/containers/storage/overlay/466f722b1915e605f1bce03ead7aec59eebc2cb222a4b66f156fd1ef29066596/merged major:0 minor:220 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/85326ed055447648dbf3a80a5271c8d2893e10f4282af192b37ab08c184f8908/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-231:{mountpoint:/var/lib/containers/storage/overlay/78cb8f2e4a033e0e0815e8d00e772a662ae83ca792afd355fe21dfd8d5a83624/merged major:0 minor:231 fsType:overlay blockSize:0} overlay_0-233:{mountpoint:/var/lib/containers/storage/overlay/2d2cf9ccee3f7f63e4a7747bac4668d9613f24110cc2b620bc892c570d3019a7/merged major:0 minor:233 fsType:overlay blockSize:0} overlay_0-235:{mountpoint:/var/lib/containers/storage/overlay/cdeae89ee609128403ad9359e5cc98d2d5b166bff256db02ed06b36a9b12b1f7/merged major:0 minor:235 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/b8a268ca5110df55342e4a87ac5c255c759d967684b88921a2058acb028050ed/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-246:{mountpoint:/var/lib/containers/storage/overlay/0cabcaa60a2ebab3917e688390a92b36bb5ee28a1e1714a75321be2d153ba2f4/merged major:0 minor:246 fsType:overlay blockSize:0} overlay_0-251:{mountpoint:/var/lib/containers/storage/overlay/24336369dfd25f425b7f2b44f53a9b79baa18d55be186ad83e2b83c998ecb99c/merged major:0 minor:251 fsType:overlay blockSize:0} overlay_0-275:{mountpoint:/var/lib/containers/storage/overlay/83f255f689fb2020541a6580fa04dc5d9897780d32415ff92f33871cf4c7dcf7/merged major:0 minor:275 fsType:overlay blockSize:0} overlay_0-277:{mountpoint:/var/lib/containers/storage/overlay/4cbae1a3a8f8edb6ac3e05815aff6ec73b48219d9300e37decce1e8108768dc3/merged major:0 minor:277 fsType:overlay blockSize:0} overlay_0-279:{mountpoint:/var/lib/containers/storage/overlay/73601743c76a8708ecb6fc69d137b359be9fa9d0e208d7697af12ed39c79925d/merged major:0 minor:279 fsType:overlay blockSize:0} overlay_0-281:{mountpoint:/var/lib/containers/storage/overlay/2b289d8f7a53ee398ca3ea262fa95392b3e647cce29fa4ee9904e7ee4cb6510f/merged major:0 minor:281 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/d22eb264bd89deb719c7682738e7a2dd6ae582d27e88fc4f30b87759a33a2b15/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-288:{mountpoint:/var/lib/containers/storage/overlay/fa97ecb4c0b41782c23355b23c84eec248a96eed8ce40eff732b5c470942d6e2/merged major:0 minor:288 fsType:overlay blockSize:0} overlay_0-290:{mountpoint:/var/lib/containers/storage/overlay/623eb491a0d347c47bce04c52f9af59929b8de47670c75c92a53392a3365a4ee/merged major:0 minor:290 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/0b94a756d76c4947a7002df36167d07169013bc218b11d5b4c0c9d5ffe12553f/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-301:{mountpoint:/var/lib/containers/storage/overlay/79f35028cf60703e0ea6686a986e1e1f4f5f7fa5467e568a83e0833057c075f3/merged major:0 minor:301 fsType:overlay blockSize:0} overlay_0-307:{mountpoint:/var/lib/containers/storage/overlay/a233e184a960dd219048328ef7a81f10a804580efd7f2fabec36e2092c92ce4c/merged major:0 minor:307 fsType:overlay blockSize:0} overlay_0-321:{mountpoint:/var/lib/containers/storage/overlay/b669309eae2c9b1078ac4204e07c5e879cfa300f927f2b81f734275760e08e30/merged major:0 minor:321 fsType:overlay blockSize:0} overlay_0-323:{mountpoint:/var/lib/containers/storage/overlay/f8713f244903cf9d20b62b80daa5ff8f88b3f8bbdcf8d7b62a6c20efbc1356cb/merged major:0 minor:323 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/f2ff102bc3cf38919108b4d6fc06d450e6e79f40f88e895415ebc75d51277af7/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/58da2283046985ed10c9e21e5bfb87b665b64e585581e785dce2a1ddecb510a6/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/84c97ea0c16aa906891d2b35603b38f740850506cc522afe62bed5eb15455493/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/57c6fe469f418ef0eed42dda23548e166b8d15ba37908a54d743b2031ee0e3be/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/42ecaac30e26fb8f8690e31a74c65fb2f83c67548911a5cb28abeec54b0b7987/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/54e9f03fb6e3ab24f2e7f297c6c0d7a52eb44803b55c80524fba4fd2272b31eb/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/3393616577f16609c4664929be7668069091340aa9aad97b14dbc96601e60727/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/9dcece9293d2ea7995f9b1c929c3be2de32662f42be95473bc23f19cae4a1f09/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/98e1672ef81bcf84530e57c07d1721b8d0ef20c68776d26588a749af73ea3fe7/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/657da407e77ecc52a0022ae52664cc9f05698fc0f3d0fd7ede2b77a043ac9d60/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.516985    4993 manager.go:217] Machine: {Timestamp:2026-11-03 20:39:57.516432826 +0000 UTC m=+0.061139187 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097280 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e608996836c052a1ee82039a5c577 SystemUUID:ec2e6089-9683-6c05-2a1e-e82039a5c577 BootID:26f56577-9871-47c0-863e-c4eaf49187ee Filesystems:[{Device:/var/lib/kubelet/pods/e5e0251d-3ac4-4b42-9175-39ffd668896f/volumes/kubernetes.io~projected/kube-api-access-j726r DeviceMajor:0 DeviceMinor:67 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d27d19c6586f35ebee131ffe2dc27cd671d13753bdf7c0c7e950c8f166acbf38/userdata/shm DeviceMajor:0 DeviceMinor:79 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-233 DeviceMajor:0 DeviceMinor:233 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/36a041f139a3468d577f21f05184c14a1f54e077154cd6ee468329e3808ee152/userdata/shm DeviceMajor:0 DeviceMinor:273 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-277 DeviceMajor:0 DeviceMinor:277 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-290 DeviceMajor:0 DeviceMinor:290 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/23545138cff3fd148cdd66ddd42dc06a3ad812b695acbaf50f39a9db273b658f/userdata/shm DeviceMajor:0 DeviceMinor:319 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-235 DeviceMajor:0 DeviceMinor:235 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ac0a7d762a7b28f0a3d5c76ce76eef22dd1f645edee406a56f21e1658de2918a/userdata/shm DeviceMajor:0 DeviceMinor:271 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/42903eab-e4bc-4f97-9e73-1109e9f5e7ab/volumes/kubernetes.io~projected/kube-api-access-t4jbc DeviceMajor:0 DeviceMinor:283 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-307 DeviceMajor:0 DeviceMinor:307 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/10a65b13-a9e4-4355-8bac-fab245388f16/volumes/kubernetes.io~projected/kube-api-access-kbpzw DeviceMajor:0 DeviceMinor:62 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0ffcf342-bccb-4362-b79e-8faf3d337b17/volumes/kubernetes.io~projected/kube-api-access-t5vgd DeviceMajor:0 DeviceMinor:65 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e92ecfcfeb535a68e348a0311cf7def199e9eb25188161b246dfbfc28236c9d5/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-231 DeviceMajor:0 DeviceMinor:231 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ea455e2d621c694b8e4c9b8c15b4a23909b85656c29641d9f26b7113965f5cae/userdata/shm DeviceMajor:0 DeviceMinor:267 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/737d534b-bee4-44e9-ac5d-124b25de7f72/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:317 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/32cf1fce-f61e-4870-aff9-a9171e90b09f/volumes/kubernetes.io~projected/kube-api-access-wmdh2 DeviceMajor:0 DeviceMinor:59 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e00279b084939ae0376b5e253923ddd2b2b67c4dd139b86e80849df2438b28d5/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e658bf30-634f-4db1-9b41-280a746540a8/volumes/kubernetes.io~projected/kube-api-access-dbw7c DeviceMajor:0 DeviceMinor:64 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/579eab1e7331b2200a1eb348da970ca5a5300f140c69684d3c64ce632b879e61/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/6b982774-43c6-45ad-a3e3-77b7bc3d534e/volumes/kubernetes.io~projected/kube-api-access-5lhxc DeviceMajor:0 DeviceMinor:266 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-288 DeviceMajor:0 DeviceMinor:288 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1169f5bf-4ec8-4f85-aba9-857c3a666a3f/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:56 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/124bfbdf1e4804f9ba9772a37c4c7a844ecefce618897fbf97094e9f69883a9b/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-220 DeviceMajor:0 DeviceMinor:220 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a168d8aade5dd2c4b5cfcf08cbb196fb2e610039d92d4de120ea4f66393a6a03/userdata/shm DeviceMajor:0 DeviceMinor:229 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-275 DeviceMajor:0 DeviceMinor:275 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ce777a6ba45a8a5134d45366ddf9787c04504e24c7b0575d8e061ee105c91d20/userdata/shm DeviceMajor:0 DeviceMinor:284 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-251 DeviceMajor:0 DeviceMinor:251 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1b5fff49-fcc5-419f-978f-afb7ea92e6c3/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:257 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9353a680-00bf-4d3a-8437-5a959a40a682/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:263 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/737d534b-bee4-44e9-ac5d-124b25de7f72/volumes/kubernetes.io~projected/kube-api-access-4rbjq DeviceMajor:0 DeviceMinor:318 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/11516670-b4cd-43cc-8c41-64e5bfed2b5a/volumes/kubernetes.io~projected/kube-api-access-9bffp DeviceMajor:0 DeviceMinor:61 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-301 DeviceMajor:0 DeviceMinor:301 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2da42e99af23864aab781534a38b114b3161276f81da308f8a18052b84a7b741/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1b5fff49-fcc5-419f-978f-afb7ea92e6c3/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:262 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-279 DeviceMajor:0 DeviceMinor:279 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e88f6a237b09b306079b4a9f5aa3eeca9a6fe5b518e5c10d63ddb151a39c9d3d/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43/volumes/kubernetes.io~projected/kube-api-access-qlfjt DeviceMajor:0 DeviceMinor:60 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-246 DeviceMajor:0 DeviceMinor:246 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a24e27a43277303c542a3fab007c981392e4c6cb4eca6bb9dbef0a265fbdc3c5/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/62995e99c79deae78ab49a3f98e520833e6ff056e23073fe3b09ddfc5f8a8e5d/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/c4cacc5f-ade7-47d9-af40-78cdc6e090ba/volumes/kubernetes.io~secret/tls-certificates DeviceMajor:0 DeviceMinor:225 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-281 DeviceMajor:0 DeviceMinor:281 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/737d534b-bee4-44e9-ac5d-124b25de7f72/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:312 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-321 DeviceMajor:0 DeviceMinor:321 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-323 DeviceMajor:0 DeviceMinor:323 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/04aa687fe51e5801032b5505b2ed2d459b20773723b08d7fefc0815ce364e082/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1169f5bf-4ec8-4f85-aba9-857c3a666a3f/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b6fe25b0-03dc-434d-957b-5f25c8d5c141/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:57 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-141 DeviceMajor:0 DeviceMinor:141 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/131affdb-a6b8-457d-84b6-ffceca9f2bfd/volumes/kubernetes.io~projected/kube-api-access-hkzsl DeviceMajor:0 DeviceMinor:203 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/264d7531f8500f215d53e74449911b84b203f7bbeb4549e3f9ab721c1d5ce85f/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/57913479-90b9-4db4-90f1-81cb81c2e5bb/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:264 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1b5fff49-fcc5-419f-978f-afb7ea92e6c3/volumes/kubernetes.io~projected/kube-api-access-5nqjx DeviceMajor:0 DeviceMinor:265 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/11516670-b4cd-43cc-8c41-64e5bfed2b5a/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:58 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1169f5bf-4ec8-4f85-aba9-857c3a666a3f/volumes/kubernetes.io~projected/kube-api-access-sl9lr DeviceMajor:0 DeviceMinor:66 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d721f687bcff21f43e7a90eb4e2129c0f46ee36289c481e438e4e3cabbdc9ff3/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/11516670-b4cd-43cc-8c41-64e5bfed2b5a/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/346aebd9-354c-4683-9eff-58058e9919c7/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:197 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2/volumes/kubernetes.io~projected/kube-api-access-nrcsl DeviceMajor:0 DeviceMinor:202 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3c2c67ac7e6f40b18ce3cf9702b813f95c87ac950a1e17a2c3ee8347bf27bc2b/userdata/shm DeviceMajor:0 DeviceMinor:269 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/57913479-90b9-4db4-90f1-81cb81c2e5bb/volumes/kubernetes.io~projected/kube-api-access-87s22 DeviceMajor:0 DeviceMinor:63 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/131affdb-a6b8-457d-84b6-ffceca9f2bfd/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:201 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/346aebd9-354c-4683-9eff-58058e9919c7/volumes/kubernetes.io~projected/kube-api-access-2f6kw DeviceMajor:0 DeviceMinor:204 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/596edf3c444e2d280e16329d46079439a81596ec44f3107fc1442753f79e2b8d/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:2da42e99af23864 MacAddress:6e:6b:76:1a:2c:50 Speed:10000 Mtu:8901} {Name:36a041f139a3468 MacAddress:5a:f6:79:4e:0e:78 Speed:10000 Mtu:8901} {Name:3c2c67ac7e6f40b MacAddress:0e:68:a7:2c:a2:93 Speed:10000 Mtu:8901} {Name:596edf3c444e2d2 MacAddress:ba:db:90:66:9c:5e Speed:10000 Mtu:8901} {Name:a168d8aade5dd2c MacAddress:52:d2:fc:50:3b:74 Speed:10000 Mtu:8901} {Name:ac0a7d762a7b28f MacAddress:a6:e5:f8:76:e5:57 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:15:0d:20:47:25 Speed:0 Mtu:9001} {Name:br-int MacAddress:66:12:1d:67:0e:a9 Speed:0 Mtu:8901} {Name:ce777a6ba45a8a5 MacAddress:0e:c0:64:dc:a3:1a Speed:10000 Mtu:8901} {Name:e00279b084939ae MacAddress:8a:83:12:5c:75:c4 Speed:10000 Mtu:8901} {Name:ea455e2d621c694 MacAddress:f2:a0:a0:9a:36:ce Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:15:0d:20:47:25 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:52:c9:bb:dd:86:d0 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:86:95:28:d3:b6:a4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.517271    4993 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.517312    4993 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.517574    4993 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.517599    4993 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-223.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.517738    4993 topology_manager.go:143] "Creating topology manager with none policy"
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.517746    4993 container_manager_linux.go:311] "Creating device plugin manager"
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.517755    4993 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.517775    4993 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.517834    4993 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.517975    4993 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.518078    4993 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.518109    4993 kubelet.go:493] "Attempting to sync node with API server"
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.518121    4993 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.518131    4993 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.518139    4993 kubelet.go:405] "Adding apiserver pod source"
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.518147    4993 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.518690    4993 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.518699    4993 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 20:39:57.520347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.519127    4993 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 03 20:39:57.530149 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530132    4993 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 03 20:39:57.530149 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530147    4993 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 03 20:39:57.530270 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530244    4993 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 03 20:39:57.530450 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530438    4993 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 03 20:39:57.530480 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530467    4993 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 03 20:39:57.530563 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530554    4993 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 03 20:39:57.530593 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530570    4993 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 03 20:39:57.530593 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530578    4993 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 03 20:39:57.530593 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530584    4993 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 03 20:39:57.530593 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530590    4993 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 03 20:39:57.530718 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530596    4993 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 03 20:39:57.530718 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530603    4993 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 03 20:39:57.530718 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530610    4993 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 03 20:39:57.530718 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530618    4993 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 03 20:39:57.530718 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530642    4993 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 03 20:39:57.530718 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530661    4993 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 03 20:39:57.530718 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530672    4993 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 03 20:39:57.530718 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530700    4993 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 03 20:39:57.530718 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.530707    4993 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 03 20:39:57.534000 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.533987    4993 server.go:1263] "Started kubelet"
Nov 03 20:39:57.534291 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.534051    4993 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 03 20:39:57.534291 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.534093    4993 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 03 20:39:57.534291 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.534172    4993 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 03 20:39:57.534592 ip-10-0-142-223 systemd[1]: Started Kubernetes Kubelet.
Nov 03 20:39:57.535203 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.534995    4993 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 03 20:39:57.535703 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.535687    4993 server.go:317] "Adding debug handlers to kubelet server"
Nov 03 20:39:57.539201 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.538454    4993 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 03 20:39:57.539201 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.538901    4993 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 03 20:39:57.539201 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.538958    4993 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 03 20:39:57.539201 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.538963    4993 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 03 20:39:57.539201 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.538984    4993 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-02 20:33:44 +0000 UTC" deadline="2028-07-17 22:16:52.205342416 +0000 UTC"
Nov 03 20:39:57.539201 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.539007    4993 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14929h36m54.66633973s"
Nov 03 20:39:57.539201 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:39:57.539040    4993 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:39:57.539201 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.539067    4993 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 03 20:39:57.541561 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541474    4993 factory.go:55] Registering systemd factory
Nov 03 20:39:57.541561 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541497    4993 factory.go:223] Registration of the systemd container factory successfully
Nov 03 20:39:57.541561 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541498    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ffcf342-bccb-4362-b79e-8faf3d337b17" volumeName="kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 03 20:39:57.541561 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541534    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b5fff49-fcc5-419f-978f-afb7ea92e6c3" volumeName="kubernetes.io/configmap/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-metrics-client-ca" seLinuxMountContext=""
Nov 03 20:39:57.541561 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541549    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4cd18396-1fa6-4b85-ad2f-922c2f4a58e2" volumeName="kubernetes.io/configmap/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-config-volume" seLinuxMountContext=""
Nov 03 20:39:57.541561 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541562    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e658bf30-634f-4db1-9b41-280a746540a8" volumeName="kubernetes.io/configmap/e658bf30-634f-4db1-9b41-280a746540a8-serviceca" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541575    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="346aebd9-354c-4683-9eff-58058e9919c7" volumeName="kubernetes.io/projected/346aebd9-354c-4683-9eff-58058e9919c7-kube-api-access-2f6kw" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541588    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42903eab-e4bc-4f97-9e73-1109e9f5e7ab" volumeName="kubernetes.io/projected/42903eab-e4bc-4f97-9e73-1109e9f5e7ab-kube-api-access-t4jbc" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541600    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ffcf342-bccb-4362-b79e-8faf3d337b17" volumeName="kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-cni-binary-copy" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541613    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10a65b13-a9e4-4355-8bac-fab245388f16" volumeName="kubernetes.io/configmap/10a65b13-a9e4-4355-8bac-fab245388f16-cni-binary-copy" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541649    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11516670-b4cd-43cc-8c41-64e5bfed2b5a" volumeName="kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-env-overrides" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541661    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="131affdb-a6b8-457d-84b6-ffceca9f2bfd" volumeName="kubernetes.io/secret/131affdb-a6b8-457d-84b6-ffceca9f2bfd-cert" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541673    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b5fff49-fcc5-419f-978f-afb7ea92e6c3" volumeName="kubernetes.io/projected/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-kube-api-access-5nqjx" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541685    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b5fff49-fcc5-419f-978f-afb7ea92e6c3" volumeName="kubernetes.io/secret/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-prometheus-operator-tls" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541697    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ffcf342-bccb-4362-b79e-8faf3d337b17" volumeName="kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541709    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10a65b13-a9e4-4355-8bac-fab245388f16" volumeName="kubernetes.io/projected/10a65b13-a9e4-4355-8bac-fab245388f16-kube-api-access-kbpzw" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541727    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1169f5bf-4ec8-4f85-aba9-857c3a666a3f" volumeName="kubernetes.io/projected/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-kube-api-access-sl9lr" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541754    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="131affdb-a6b8-457d-84b6-ffceca9f2bfd" volumeName="kubernetes.io/projected/131affdb-a6b8-457d-84b6-ffceca9f2bfd-kube-api-access-hkzsl" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541766    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="346aebd9-354c-4683-9eff-58058e9919c7" volumeName="kubernetes.io/configmap/346aebd9-354c-4683-9eff-58058e9919c7-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541778    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="346aebd9-354c-4683-9eff-58058e9919c7" volumeName="kubernetes.io/empty-dir/346aebd9-354c-4683-9eff-58058e9919c7-data-volume" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541790    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="737d534b-bee4-44e9-ac5d-124b25de7f72" volumeName="kubernetes.io/secret/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541802    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9353a680-00bf-4d3a-8437-5a959a40a682" volumeName="kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541816    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11516670-b4cd-43cc-8c41-64e5bfed2b5a" volumeName="kubernetes.io/secret/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541828    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1169f5bf-4ec8-4f85-aba9-857c3a666a3f" volumeName="kubernetes.io/empty-dir/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-tmp" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541839    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1b5fff49-fcc5-419f-978f-afb7ea92e6c3" volumeName="kubernetes.io/secret/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541852    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4cd18396-1fa6-4b85-ad2f-922c2f4a58e2" volumeName="kubernetes.io/projected/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-kube-api-access-nrcsl" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541864    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11516670-b4cd-43cc-8c41-64e5bfed2b5a" volumeName="kubernetes.io/projected/11516670-b4cd-43cc-8c41-64e5bfed2b5a-kube-api-access-9bffp" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541876    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="32cf1fce-f61e-4870-aff9-a9171e90b09f" volumeName="kubernetes.io/projected/32cf1fce-f61e-4870-aff9-a9171e90b09f-kube-api-access-wmdh2" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541889    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6b982774-43c6-45ad-a3e3-77b7bc3d534e" volumeName="kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541902    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="737d534b-bee4-44e9-ac5d-124b25de7f72" volumeName="kubernetes.io/configmap/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541918    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="737d534b-bee4-44e9-ac5d-124b25de7f72" volumeName="kubernetes.io/projected/737d534b-bee4-44e9-ac5d-124b25de7f72-kube-api-access-4rbjq" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541930    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e5e0251d-3ac4-4b42-9175-39ffd668896f" volumeName="kubernetes.io/configmap/e5e0251d-3ac4-4b42-9175-39ffd668896f-iptables-alerter-script" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541942    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e5e0251d-3ac4-4b42-9175-39ffd668896f" volumeName="kubernetes.io/projected/e5e0251d-3ac4-4b42-9175-39ffd668896f-kube-api-access-j726r" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541956    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42903eab-e4bc-4f97-9e73-1109e9f5e7ab" volumeName="kubernetes.io/empty-dir/42903eab-e4bc-4f97-9e73-1109e9f5e7ab-tmp" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541969    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10a65b13-a9e4-4355-8bac-fab245388f16" volumeName="kubernetes.io/configmap/10a65b13-a9e4-4355-8bac-fab245388f16-multus-daemon-config" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541982    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11516670-b4cd-43cc-8c41-64e5bfed2b5a" volumeName="kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovnkube-config" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.541994    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4cd18396-1fa6-4b85-ad2f-922c2f4a58e2" volumeName="kubernetes.io/empty-dir/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-tmp-dir" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542007    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4cd18396-1fa6-4b85-ad2f-922c2f4a58e2" volumeName="kubernetes.io/secret/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-metrics-tls" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542018    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="737d534b-bee4-44e9-ac5d-124b25de7f72" volumeName="kubernetes.io/empty-dir/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-textfile" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542030    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="737d534b-bee4-44e9-ac5d-124b25de7f72" volumeName="kubernetes.io/secret/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-tls" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542044    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e658bf30-634f-4db1-9b41-280a746540a8" volumeName="kubernetes.io/projected/e658bf30-634f-4db1-9b41-280a746540a8-kube-api-access-dbw7c" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542056    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0ffcf342-bccb-4362-b79e-8faf3d337b17" volumeName="kubernetes.io/projected/0ffcf342-bccb-4362-b79e-8faf3d337b17-kube-api-access-t5vgd" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542069    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11516670-b4cd-43cc-8c41-64e5bfed2b5a" volumeName="kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovnkube-script-lib" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542080    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="346aebd9-354c-4683-9eff-58058e9919c7" volumeName="kubernetes.io/secret/346aebd9-354c-4683-9eff-58058e9919c7-insights-runtime-extractor-tls" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542092    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="57913479-90b9-4db4-90f1-81cb81c2e5bb" volumeName="kubernetes.io/projected/57913479-90b9-4db4-90f1-81cb81c2e5bb-kube-api-access-87s22" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542103    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="57913479-90b9-4db4-90f1-81cb81c2e5bb" volumeName="kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542116    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="853d1e90-3b6e-4bb5-b6c5-82b1ce141e43" volumeName="kubernetes.io/projected/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-kube-api-access-qlfjt" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542160    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6fe25b0-03dc-434d-957b-5f25c8d5c141" volumeName="kubernetes.io/configmap/b6fe25b0-03dc-434d-957b-5f25c8d5c141-konnectivity-ca" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542174    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4cacc5f-ade7-47d9-af40-78cdc6e090ba" volumeName="kubernetes.io/secret/c4cacc5f-ade7-47d9-af40-78cdc6e090ba-tls-certificates" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542186    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1169f5bf-4ec8-4f85-aba9-857c3a666a3f" volumeName="kubernetes.io/empty-dir/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-tuned" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542197    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="737d534b-bee4-44e9-ac5d-124b25de7f72" volumeName="kubernetes.io/configmap/737d534b-bee4-44e9-ac5d-124b25de7f72-metrics-client-ca" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542208    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="853d1e90-3b6e-4bb5-b6c5-82b1ce141e43" volumeName="kubernetes.io/empty-dir/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-tmp-dir" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542220    4993 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6fe25b0-03dc-434d-957b-5f25c8d5c141" volumeName="kubernetes.io/secret/b6fe25b0-03dc-434d-957b-5f25c8d5c141-agent-certs" seLinuxMountContext=""
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542231    4993 reconstruct.go:97] "Volume reconstruction finished"
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542238    4993 reconciler.go:29] "Reconciler: start to sync state"
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.542993    4993 factory.go:153] Registering CRI-O factory
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.543010    4993 factory.go:223] Registration of the crio container factory successfully
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.543063    4993 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.543079    4993 factory.go:103] Registering Raw factory
Nov 03 20:39:57.543308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.543095    4993 manager.go:1199] Started watching for new ooms in manager
Nov 03 20:39:57.545600 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.543472    4993 manager.go:319] Starting recovery of all containers
Nov 03 20:39:57.568014 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.567980    4993 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 03 20:39:57.569496 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.569478    4993 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 03 20:39:57.569496 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.569500    4993 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 03 20:39:57.569658 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.569516    4993 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 03 20:39:57.569658 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:39:57.569547    4993 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 03 20:39:57.585709 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.585681    4993 generic.go:356] "Generic (PLEG): container finished" podID="61bc8eba7840edd90e06bc583d329627" containerID="93682a4a249d3236e42950bc5d86fcc9d047fd28b75663925647015447c9e425" exitCode=0
Nov 03 20:39:57.613100 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.613068    4993 generic.go:356] "Generic (PLEG): container finished" podID="0ffcf342-bccb-4362-b79e-8faf3d337b17" containerID="99bcd51319ec2acd93b17eb9747949fb2230cd212c3ee200a3155c813661b322" exitCode=0
Nov 03 20:39:57.613100 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.613100    4993 generic.go:356] "Generic (PLEG): container finished" podID="0ffcf342-bccb-4362-b79e-8faf3d337b17" containerID="e84b98cd65ad35bfa8543281d6908a65963eaaafbf82050e65f4124448cb257b" exitCode=0
Nov 03 20:39:57.613278 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.613112    4993 generic.go:356] "Generic (PLEG): container finished" podID="0ffcf342-bccb-4362-b79e-8faf3d337b17" containerID="46430392033ef2e3c328d1a3386bf50e81610fb104d080ac7eb5ad4e340c39cd" exitCode=0
Nov 03 20:39:57.613278 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.613121    4993 generic.go:356] "Generic (PLEG): container finished" podID="0ffcf342-bccb-4362-b79e-8faf3d337b17" containerID="6f84edd7aec47fab4920eb5c23965afdaec9c42e9f01dfd3b8d930a0e00a0862" exitCode=0
Nov 03 20:39:57.613278 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.613130    4993 generic.go:356] "Generic (PLEG): container finished" podID="0ffcf342-bccb-4362-b79e-8faf3d337b17" containerID="264da65c804b5693f6c000f725034c2aa12e387ca7284e7ed7426ac0155d1e63" exitCode=0
Nov 03 20:39:57.613278 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.613139    4993 generic.go:356] "Generic (PLEG): container finished" podID="0ffcf342-bccb-4362-b79e-8faf3d337b17" containerID="61c7c63d69cd5233d52a1985b96ddaa16a3bb3d58b9a098a3ffbddd343a094fd" exitCode=0
Nov 03 20:39:57.622469 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.622454    4993 manager.go:324] Recovery completed
Nov 03 20:39:57.638861 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.638805    4993 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:39:57.640100 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:39:57.640083    4993 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:39:57.642702 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.642684    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:39:57.642777 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.642722    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:39:57.642777 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.642735    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:39:57.645180 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.645162    4993 cpu_manager.go:226] "Starting" policy="none"
Nov 03 20:39:57.645180 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.645179    4993 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 03 20:39:57.645311 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.645180    4993 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:39:57.645311 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.645198    4993 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 03 20:39:57.645397 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.645316    4993 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 03 20:39:57.645397 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.645327    4993 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 03 20:39:57.645397 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.645342    4993 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 03 20:39:57.645397 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.645347    4993 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 03 20:39:57.645397 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.645352    4993 policy_none.go:50] "Start"
Nov 03 20:39:57.645397 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.645360    4993 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 03 20:39:57.645397 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.645369    4993 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 03 20:39:57.645609 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.645484    4993 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 03 20:39:57.645609 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.645492    4993 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 03 20:39:57.645609 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.645497    4993 policy_none.go:44] "Start"
Nov 03 20:39:57.646092 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.646073    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:39:57.646189 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.646103    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:39:57.646189 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.646129    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:39:57.649745 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.649718    4993 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 03 20:39:57.655022 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.655008    4993 manager.go:341] "Starting Device Plugin manager"
Nov 03 20:39:57.655091 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:39:57.655032    4993 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 03 20:39:57.655091 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.655040    4993 server.go:87] "Starting device plugin registration server"
Nov 03 20:39:57.655327 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.655315    4993 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 03 20:39:57.655358 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.655330    4993 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 03 20:39:57.655418 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.655403    4993 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 03 20:39:57.655537 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.655508    4993 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 03 20:39:57.655604 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.655545    4993 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 03 20:39:57.655604 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.655552    4993 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 03 20:39:57.655693 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.655600    4993 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-03T20:39:57.655536116Z","UUID":"902df397-5e6a-43dc-9e46-1d603e09b02c","Handler":null,"Name":"","Endpoint":""}
Nov 03 20:39:57.656915 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.656888    4993 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 03 20:39:57.656915 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.656910    4993 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 03 20:39:57.657073 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:39:57.656917    4993 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 03 20:39:57.657073 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:39:57.656943    4993 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-223.us-west-1.compute.internal\" not found"
Nov 03 20:39:57.670181 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.670146    4993 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal"]
Nov 03 20:39:57.670282 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.670270    4993 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:39:57.671217 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.671194    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:39:57.671217 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.671219    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:39:57.671334 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.671229    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:39:57.671467 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.671456    4993 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:39:57.671594 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.671579    4993 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:39:57.672168 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.672152    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:39:57.672267 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.672183    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:39:57.672267 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.672152    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:39:57.672267 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.672226    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:39:57.672267 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.672194    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:39:57.672267 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.672248    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:39:57.672499 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:39:57.672414    4993 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found" node="ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.672499 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.672459    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal" event={"ID":"61bc8eba7840edd90e06bc583d329627","Type":"ContainerStarted","Data":"eaa33074687f321d51423a81a98a26e966dcf6222407d8b73e06680d942001fd"}
Nov 03 20:39:57.672499 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.672487    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal" event={"ID":"61bc8eba7840edd90e06bc583d329627","Type":"ContainerDied","Data":"93682a4a249d3236e42950bc5d86fcc9d047fd28b75663925647015447c9e425"}
Nov 03 20:39:57.672617 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.672505    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal" event={"ID":"61bc8eba7840edd90e06bc583d329627","Type":"ContainerStarted","Data":"e88f6a237b09b306079b4a9f5aa3eeca9a6fe5b518e5c10d63ddb151a39c9d3d"}
Nov 03 20:39:57.672617 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.672524    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal" event={"ID":"88d138da692bc0e2c453d9d2e3886bfc","Type":"ContainerStarted","Data":"98e4b4699033c16352e758d565a30a64fbc9b98a10883c80e7223fc8fc21d6b1"}
Nov 03 20:39:57.672617 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.672535    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal" event={"ID":"88d138da692bc0e2c453d9d2e3886bfc","Type":"ContainerStarted","Data":"04aa687fe51e5801032b5505b2ed2d459b20773723b08d7fefc0815ce364e082"}
Nov 03 20:39:57.672617 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.672555    4993 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:39:57.673429 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.673415    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:39:57.673512 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.673446    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:39:57.673512 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.673461    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:39:57.673622 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:39:57.673604    4993 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-223.us-west-1.compute.internal\" not found" node="ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.730704 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.730679    4993 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:39:57.731754 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.731738    4993 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 03 20:39:57.731821 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.731784    4993 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:39:57.740078 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.740065    4993 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.756300 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.756279    4993 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 03 20:39:57.757208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.757193    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 03 20:39:57.757303 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.757224    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 03 20:39:57.757303 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.757240    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 03 20:39:57.757303 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.757285    4993 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.779160 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.779135    4993 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:39:57.829521 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.829498    4993 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 20:39:57.829606 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.829531    4993 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 03 20:39:57.829606 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:39:57.829551    4993 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.829606 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.829566    4993 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.832794 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.832778    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/88d138da692bc0e2c453d9d2e3886bfc-config\") pod \"kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"88d138da692bc0e2c453d9d2e3886bfc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.832841 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.832801    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61bc8eba7840edd90e06bc583d329627-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"61bc8eba7840edd90e06bc583d329627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.832841 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.832820    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61bc8eba7840edd90e06bc583d329627-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"61bc8eba7840edd90e06bc583d329627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.883965 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.883944    4993 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.884034 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.883993    4993 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.884373 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.884358    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeNotReady"
Nov 03 20:39:57.884433 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.884384    4993 setters.go:546] "Node became not ready" node="ip-10-0-142-223.us-west-1.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-03T20:39:57Z","lastTransitionTime":"2026-11-03T20:39:57Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Nov 03 20:39:57.901448 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.901407    4993 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 20:39:57.901448 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:39:57.901439    4993 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.933583 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.933558    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61bc8eba7840edd90e06bc583d329627-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"61bc8eba7840edd90e06bc583d329627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.933690 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.933587    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61bc8eba7840edd90e06bc583d329627-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"61bc8eba7840edd90e06bc583d329627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.933690 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.933669    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/88d138da692bc0e2c453d9d2e3886bfc-config\") pod \"kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"88d138da692bc0e2c453d9d2e3886bfc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.933690 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.933678    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61bc8eba7840edd90e06bc583d329627-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"61bc8eba7840edd90e06bc583d329627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.933690 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.933679    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/61bc8eba7840edd90e06bc583d329627-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"61bc8eba7840edd90e06bc583d329627\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.933837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.933708    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/88d138da692bc0e2c453d9d2e3886bfc-config\") pod \"kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal\" (UID: \"88d138da692bc0e2c453d9d2e3886bfc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:57.943478 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:57.943447    4993 nodeinfomanager.go:471] existing CSINode "ip-10-0-142-223.us-west-1.compute.internal" is owned by different node (oldNodeID="1b0d20da-5684-4d68-ad0e-ee56843396cb", newNodeID=""), cleaning up...
Nov 03 20:39:58.519245 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.519217    4993 apiserver.go:52] "Watching apiserver"
Nov 03 20:39:58.619083 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.619058    4993 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:58.619233 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.619220    4993 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:58.622754 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.622736    4993 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 03 20:39:58.623231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.623211    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal","openshift-console/downloads-6dd789c8b7-hd6z4","openshift-multus/multus-585md","openshift-multus/network-metrics-daemon-hmnm7","kube-system/konnectivity-agent-h952v","openshift-cluster-node-tuning-operator/tuned-dm2zp","openshift-dns/dns-default-45x6l","openshift-monitoring/thanos-querier-d45496948-n4hjf","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h","kube-system/global-pull-secret-syncer-8jhcx","openshift-dns/node-resolver-pbhv8","openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt","openshift-multus/multus-additional-cni-plugins-lxzlf","openshift-insights/insights-runtime-extractor-j86dg","openshift-network-diagnostics/network-check-target-l9p4t","openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt","openshift-ingress-canary/ingress-canary-kmzjf","openshift-monitoring/node-exporter-6sjbh","openshift-image-registry/node-ca-9hqmv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal","openshift-network-operator/iptables-alerter-4dwkb","openshift-ovn-kubernetes/ovnkube-node-pwvcf"]
Nov 03 20:39:58.635069 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.635049    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.641037 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.641018    4993 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 03 20:39:58.677922 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.677896    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 03 20:39:58.677922 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.677906    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 03 20:39:58.680831 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.680810    4993 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 20:39:58.680917 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:39:58.680901    4993 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:58.681107 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.681091    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 03 20:39:58.681175 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.681121    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 03 20:39:58.681175 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.681132    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 03 20:39:58.681319 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.681305    4993 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 03 20:39:58.681375 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:39:58.681354    4993 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-223.us-west-1.compute.internal"
Nov 03 20:39:58.681472 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.681458    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 03 20:39:58.682440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.682423    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dm4wl\""
Nov 03 20:39:58.682510 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.682443    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 03 20:39:58.682618 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.682599    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 03 20:39:58.682702 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.682603    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-mtgd7\""
Nov 03 20:39:58.682702 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.682667    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-czwwq\""
Nov 03 20:39:58.682702 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.682675    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-29dtl\""
Nov 03 20:39:58.682702 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.682677    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8nf42\""
Nov 03 20:39:58.682702 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.682693    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 03 20:39:58.682889 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.682694    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r6k78\""
Nov 03 20:39:58.682889 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.682671    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 03 20:39:58.682889 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.682728    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nnzhp\""
Nov 03 20:39:58.682889 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.682755    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-kqp9s\""
Nov 03 20:39:58.682889 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.682793    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 03 20:39:58.682889 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.682833    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zxrmv\""
Nov 03 20:39:58.682889 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.682883    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4xfms\""
Nov 03 20:39:58.683145 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.683123    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j2mnh\""
Nov 03 20:39:58.703565 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.703545    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zvv9n\""
Nov 03 20:39:58.716251 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.716229    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 03 20:39:58.716519 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.716504    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 03 20:39:58.716776 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.716758    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 03 20:39:58.716848 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.716793    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 03 20:39:58.716848 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.716818    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 03 20:39:58.716942 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.716873    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 03 20:39:58.716942 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.716885    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 03 20:39:58.717043 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.717029    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 03 20:39:58.717107 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.717040    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:58.717107 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.717067    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 03 20:39:58.717107 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.717076    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 03 20:39:58.717107 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.717080    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 03 20:39:58.717107 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.717102    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:39:58.717268 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.717081    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 03 20:39:58.717268 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.717079    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 03 20:39:58.717268 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.717116    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 03 20:39:58.717268 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.717137    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 03 20:39:58.717372 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.717300    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 03 20:39:58.717420 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.717410    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 03 20:39:58.717599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.717586    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 03 20:39:58.722541 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.722520    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 03 20:39:58.722541 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.722534    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 03 20:39:58.722818 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.722804    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 03 20:39:58.722872 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.722839    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 03 20:39:58.722872 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.722849    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 03 20:39:58.722872 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.722863    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 03 20:39:58.722872 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.722870    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 03 20:39:58.723041 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.722851    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 03 20:39:58.723041 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.722884    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 03 20:39:58.723041 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.722916    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 03 20:39:58.723041 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.722925    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 03 20:39:58.723041 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.722968    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 03 20:39:58.737460 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737422    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-cnibin\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.737460 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737450    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/131affdb-a6b8-457d-84b6-ffceca9f2bfd-cert\") pod \"ingress-canary-kmzjf\" (UID: \"131affdb-a6b8-457d-84b6-ffceca9f2bfd\") " pod="openshift-ingress-canary/ingress-canary-kmzjf"
Nov 03 20:39:58.737604 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737470    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-cni-binary-copy\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.737604 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737488    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.737604 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737522    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5e0251d-3ac4-4b42-9175-39ffd668896f-host-slash\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:39:58.737604 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737554    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-slash\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.737780 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737613    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-tmp\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.737780 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737722    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-grpc-tls\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.737780 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737762    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-etc-kubernetes\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.737885 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737790    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-cni-binary-copy\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.737885 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737830    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovnkube-config\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.737885 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737859    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-env-overrides\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.737987 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737884    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4jbc\" (UniqueName: \"kubernetes.io/projected/42903eab-e4bc-4f97-9e73-1109e9f5e7ab-kube-api-access-t4jbc\") pod \"downloads-6dd789c8b7-hd6z4\" (UID: \"42903eab-e4bc-4f97-9e73-1109e9f5e7ab\") " pod="openshift-console/downloads-6dd789c8b7-hd6z4"
Nov 03 20:39:58.737987 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737910    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl9lr\" (UniqueName: \"kubernetes.io/projected/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-kube-api-access-sl9lr\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.737987 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737958    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-wtmp\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.737987 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.737983    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nqjx\" (UniqueName: \"kubernetes.io/projected/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-kube-api-access-5nqjx\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:58.738181 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738006    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovnkube-config\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.738181 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738013    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e658bf30-634f-4db1-9b41-280a746540a8-host\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:39:58.738181 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738063    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-env-overrides\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.738181 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738115    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovnkube-script-lib\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.738181 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738142    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.738181 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738171    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.738440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738204    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-cni-netd\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.738440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738231    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:58.738440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738273    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-tmp-dir\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:39:58.738440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738308    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-run-netns\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.738440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738339    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87s22\" (UniqueName: \"kubernetes.io/projected/57913479-90b9-4db4-90f1-81cb81c2e5bb-kube-api-access-87s22\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:58.738440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738354    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.738440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738368    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-tmp-dir\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:39:58.738440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738378    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-system-cni-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.738440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738383    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovnkube-script-lib\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.738440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738417    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-cni-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738455    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738486    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c4cacc5f-ade7-47d9-af40-78cdc6e090ba-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-gwzjt\" (UID: \"c4cacc5f-ade7-47d9-af40-78cdc6e090ba\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738492    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57913479-90b9-4db4-90f1-81cb81c2e5bb-metrics-certs\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738511    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738538    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-kubelet\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738558    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-log-socket\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738573    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-socket-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738602    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-thanos-querier-tls\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738640    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738684    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738718    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-metrics-client-ca\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738767    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-cni-multus\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738788    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9353a680-00bf-4d3a-8437-5a959a40a682-original-pull-secret\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738792    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738817    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42903eab-e4bc-4f97-9e73-1109e9f5e7ab-tmp\") pod \"downloads-6dd789c8b7-hd6z4\" (UID: \"42903eab-e4bc-4f97-9e73-1109e9f5e7ab\") " pod="openshift-console/downloads-6dd789c8b7-hd6z4"
Nov 03 20:39:58.738837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738831    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-sys\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738846    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738867    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhxc\" (UniqueName: \"kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc\") pod \"network-check-target-l9p4t\" (UID: \"6b982774-43c6-45ad-a3e3-77b7bc3d534e\") " pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738899    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbw7c\" (UniqueName: \"kubernetes.io/projected/e658bf30-634f-4db1-9b41-280a746540a8-kube-api-access-dbw7c\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.738904    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42903eab-e4bc-4f97-9e73-1109e9f5e7ab-tmp\") pod \"downloads-6dd789c8b7-hd6z4\" (UID: \"42903eab-e4bc-4f97-9e73-1109e9f5e7ab\") " pod="openshift-console/downloads-6dd789c8b7-hd6z4"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739005    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-etc-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739108    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-root\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739144    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-cnibin\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739183    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-socket-dir-parent\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739218    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-k8s-cni-cncf-io\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739249    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10a65b13-a9e4-4355-8bac-fab245388f16-multus-daemon-config\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739287    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-multus-certs\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739382    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-textfile\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739410    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-netns\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739434    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/346aebd9-354c-4683-9eff-58058e9919c7-crio-socket\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739458    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-textfile\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739480    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10a65b13-a9e4-4355-8bac-fab245388f16-multus-daemon-config\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739492    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.739554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739522    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysconfig\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739563    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-sys-fs\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739587    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-kubernetes\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739605    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rbjq\" (UniqueName: \"kubernetes.io/projected/737d534b-bee4-44e9-ac5d-124b25de7f72-kube-api-access-4rbjq\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739644    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbpzw\" (UniqueName: \"kubernetes.io/projected/10a65b13-a9e4-4355-8bac-fab245388f16-kube-api-access-kbpzw\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739718    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-cni-bin\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739812    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-config-volume\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739876    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-ovn\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739899    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739928    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-system-cni-dir\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739939    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-config-volume\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.739978    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-tmp\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740059    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-tmp\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740084    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkzsl\" (UniqueName: \"kubernetes.io/projected/131affdb-a6b8-457d-84b6-ffceca9f2bfd-kube-api-access-hkzsl\") pod \"ingress-canary-kmzjf\" (UID: \"131affdb-a6b8-457d-84b6-ffceca9f2bfd\") " pod="openshift-ingress-canary/ingress-canary-kmzjf"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740107    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-cni-bin\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740122    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-modprobe-d\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740138    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-lib-modules\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740165    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9353a680-00bf-4d3a-8437-5a959a40a682-kubelet-config\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:58.740208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740204    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740239    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740296    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-systemd\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740330    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-registration-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740356    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740371    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-sys\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740401    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10a65b13-a9e4-4355-8bac-fab245388f16-cni-binary-copy\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740431    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-hostroot\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740492    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740517    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlfjt\" (UniqueName: \"kubernetes.io/projected/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-kube-api-access-qlfjt\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740570    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5e0251d-3ac4-4b42-9175-39ffd668896f-iptables-alerter-script\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740592    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovn-node-metrics-cert\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740614    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/346aebd9-354c-4683-9eff-58058e9919c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740622    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10a65b13-a9e4-4355-8bac-fab245388f16-cni-binary-copy\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740655    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jqb9\" (UniqueName: \"kubernetes.io/projected/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-kube-api-access-9jqb9\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740701    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ffcf342-bccb-4362-b79e-8faf3d337b17-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740740    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-hosts-file\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740768    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-etc-selinux\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740795    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-systemd\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740817    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5e0251d-3ac4-4b42-9175-39ffd668896f-iptables-alerter-script\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740834    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrcsl\" (UniqueName: \"kubernetes.io/projected/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-kube-api-access-nrcsl\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740856    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/346aebd9-354c-4683-9eff-58058e9919c7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740866    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bffp\" (UniqueName: \"kubernetes.io/projected/11516670-b4cd-43cc-8c41-64e5bfed2b5a-kube-api-access-9bffp\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740893    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysctl-d\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740909    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/11516670-b4cd-43cc-8c41-64e5bfed2b5a-ovn-node-metrics-cert\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740920    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-tmp-dir\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:58.740963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740954    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-systemd-units\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740991    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/346aebd9-354c-4683-9eff-58058e9919c7-data-volume\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.740999    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-tmp-dir\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741026    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-tls\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741048    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-var-lib-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741057    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/346aebd9-354c-4683-9eff-58058e9919c7-data-volume\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741069    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/737d534b-bee4-44e9-ac5d-124b25de7f72-metrics-client-ca\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741088    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f6kw\" (UniqueName: \"kubernetes.io/projected/346aebd9-354c-4683-9eff-58058e9919c7-kube-api-access-2f6kw\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741109    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysctl-conf\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741123    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-run\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741177    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-metrics-tls\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741237    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmdh2\" (UniqueName: \"kubernetes.io/projected/32cf1fce-f61e-4870-aff9-a9171e90b09f-kube-api-access-wmdh2\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741273    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741301    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e658bf30-634f-4db1-9b41-280a746540a8-serviceca\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741367    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/346aebd9-354c-4683-9eff-58058e9919c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741402    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5vgd\" (UniqueName: \"kubernetes.io/projected/0ffcf342-bccb-4362-b79e-8faf3d337b17-kube-api-access-t5vgd\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741428    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-tuned\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741439    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741455    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b6fe25b0-03dc-434d-957b-5f25c8d5c141-agent-certs\") pod \"konnectivity-agent-h952v\" (UID: \"b6fe25b0-03dc-434d-957b-5f25c8d5c141\") " pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741480    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-device-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741508    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-var-lib-kubelet\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741515    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-tuned\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741535    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741570    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j726r\" (UniqueName: \"kubernetes.io/projected/e5e0251d-3ac4-4b42-9175-39ffd668896f-kube-api-access-j726r\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741683    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-node-log\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741719    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-host\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741745    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/737d534b-bee4-44e9-ac5d-124b25de7f72-metrics-client-ca\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741753    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b6fe25b0-03dc-434d-957b-5f25c8d5c141-agent-certs\") pod \"konnectivity-agent-h952v\" (UID: \"b6fe25b0-03dc-434d-957b-5f25c8d5c141\") " pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741747    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b6fe25b0-03dc-434d-957b-5f25c8d5c141-konnectivity-ca\") pod \"konnectivity-agent-h952v\" (UID: \"b6fe25b0-03dc-434d-957b-5f25c8d5c141\") " pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741774    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-accelerators-collector-config\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741855    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9353a680-00bf-4d3a-8437-5a959a40a682-dbus\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741892    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-kubelet\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741913    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-conf-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741932    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e658bf30-634f-4db1-9b41-280a746540a8-serviceca\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:39:58.742058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.741958    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b6fe25b0-03dc-434d-957b-5f25c8d5c141-konnectivity-ca\") pod \"konnectivity-agent-h952v\" (UID: \"b6fe25b0-03dc-434d-957b-5f25c8d5c141\") " pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:39:58.781775 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.781719    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Nov 03 20:39:58.781974 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.781957    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Nov 03 20:39:58.782045 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.781993    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Nov 03 20:39:58.782045 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.782024    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 03 20:39:58.782045 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.782028    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-fjpbj\""
Nov 03 20:39:58.782229 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.782214    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Nov 03 20:39:58.782279 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.782241    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Nov 03 20:39:58.782279 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.782247    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-sj26z\""
Nov 03 20:39:58.782279 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.782247    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 03 20:39:58.782441 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.782422    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rpkqt\""
Nov 03 20:39:58.783186 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.782845    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 03 20:39:58.783186 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.782848    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Nov 03 20:39:58.783186 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.782996    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9ll5z\""
Nov 03 20:39:58.783186 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.783035    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fi791uldhbe9d\""
Nov 03 20:39:58.783186 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.783048    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Nov 03 20:39:58.783186 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.783139    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rzwv7\""
Nov 03 20:39:58.783477 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.783196    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Nov 03 20:39:58.783477 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.783201    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 03 20:39:58.783477 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.783393    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-l4h2k\""
Nov 03 20:39:58.783477 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.783414    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-glsfk\""
Nov 03 20:39:58.783665 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.783652    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9fj5k\""
Nov 03 20:39:58.783735 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.783668    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 03 20:39:58.788063 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.788044    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.788142 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.788064    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/131affdb-a6b8-457d-84b6-ffceca9f2bfd-cert\") pod \"ingress-canary-kmzjf\" (UID: \"131affdb-a6b8-457d-84b6-ffceca9f2bfd\") " pod="openshift-ingress-canary/ingress-canary-kmzjf"
Nov 03 20:39:58.788755 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.788738    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c4cacc5f-ade7-47d9-af40-78cdc6e090ba-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-gwzjt\" (UID: \"c4cacc5f-ade7-47d9-af40-78cdc6e090ba\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt"
Nov 03 20:39:58.788908 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.788895    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:58.791084 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.791070    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:58.791220 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.791207    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-tls\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.792187 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.792174    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-metrics-tls\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:58.792229 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.792175    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/346aebd9-354c-4683-9eff-58058e9919c7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:58.796361 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.796342    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl9lr\" (UniqueName: \"kubernetes.io/projected/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-kube-api-access-sl9lr\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.797809 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.797790    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87s22\" (UniqueName: \"kubernetes.io/projected/57913479-90b9-4db4-90f1-81cb81c2e5bb-kube-api-access-87s22\") pod \"network-metrics-daemon-hmnm7\" (UID: \"57913479-90b9-4db4-90f1-81cb81c2e5bb\") " pod="openshift-multus/network-metrics-daemon-hmnm7"
Nov 03 20:39:58.798577 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.798556    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4jbc\" (UniqueName: \"kubernetes.io/projected/42903eab-e4bc-4f97-9e73-1109e9f5e7ab-kube-api-access-t4jbc\") pod \"downloads-6dd789c8b7-hd6z4\" (UID: \"42903eab-e4bc-4f97-9e73-1109e9f5e7ab\") " pod="openshift-console/downloads-6dd789c8b7-hd6z4"
Nov 03 20:39:58.798868 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.798852    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nqjx\" (UniqueName: \"kubernetes.io/projected/1b5fff49-fcc5-419f-978f-afb7ea92e6c3-kube-api-access-5nqjx\") pod \"prometheus-operator-9cd89b74c-8gqtt\" (UID: \"1b5fff49-fcc5-419f-978f-afb7ea92e6c3\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-8gqtt"
Nov 03 20:39:58.800981 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.800965    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkzsl\" (UniqueName: \"kubernetes.io/projected/131affdb-a6b8-457d-84b6-ffceca9f2bfd-kube-api-access-hkzsl\") pod \"ingress-canary-kmzjf\" (UID: \"131affdb-a6b8-457d-84b6-ffceca9f2bfd\") " pod="openshift-ingress-canary/ingress-canary-kmzjf"
Nov 03 20:39:58.801747 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.801730    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlfjt\" (UniqueName: \"kubernetes.io/projected/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-kube-api-access-qlfjt\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:39:58.801973 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.801959    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lhxc\" (UniqueName: \"kubernetes.io/projected/6b982774-43c6-45ad-a3e3-77b7bc3d534e-kube-api-access-5lhxc\") pod \"network-check-target-l9p4t\" (UID: \"6b982774-43c6-45ad-a3e3-77b7bc3d534e\") " pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:39:58.802439 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.802411    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmdh2\" (UniqueName: \"kubernetes.io/projected/32cf1fce-f61e-4870-aff9-a9171e90b09f-kube-api-access-wmdh2\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.802982 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.802966    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f6kw\" (UniqueName: \"kubernetes.io/projected/346aebd9-354c-4683-9eff-58058e9919c7-kube-api-access-2f6kw\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:58.803294 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.803279    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbw7c\" (UniqueName: \"kubernetes.io/projected/e658bf30-634f-4db1-9b41-280a746540a8-kube-api-access-dbw7c\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:39:58.804014 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.803993    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5vgd\" (UniqueName: \"kubernetes.io/projected/0ffcf342-bccb-4362-b79e-8faf3d337b17-kube-api-access-t5vgd\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.804014 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.804009    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j726r\" (UniqueName: \"kubernetes.io/projected/e5e0251d-3ac4-4b42-9175-39ffd668896f-kube-api-access-j726r\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:39:58.806762 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.806741    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bffp\" (UniqueName: \"kubernetes.io/projected/11516670-b4cd-43cc-8c41-64e5bfed2b5a-kube-api-access-9bffp\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.807060 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.807040    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rbjq\" (UniqueName: \"kubernetes.io/projected/737d534b-bee4-44e9-ac5d-124b25de7f72-kube-api-access-4rbjq\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.807282 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.807268    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrcsl\" (UniqueName: \"kubernetes.io/projected/4cd18396-1fa6-4b85-ad2f-922c2f4a58e2-kube-api-access-nrcsl\") pod \"dns-default-45x6l\" (UID: \"4cd18396-1fa6-4b85-ad2f-922c2f4a58e2\") " pod="openshift-dns/dns-default-45x6l"
Nov 03 20:39:58.807761 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.807746    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbpzw\" (UniqueName: \"kubernetes.io/projected/10a65b13-a9e4-4355-8bac-fab245388f16-kube-api-access-kbpzw\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.842778 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.842755    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-cni-bin\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.842876 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.842780    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-modprobe-d\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.842876 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.842797    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-lib-modules\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.842876 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.842811    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9353a680-00bf-4d3a-8437-5a959a40a682-kubelet-config\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:58.842876 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.842827    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.843027 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.842884    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-cni-bin\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.843027 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.842900    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9353a680-00bf-4d3a-8437-5a959a40a682-kubelet-config\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:58.843027 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.842942    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-lib-modules\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.843027 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.842993    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-modprobe-d\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.843027 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843010    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-systemd\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843036    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-registration-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843052    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843066    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-sys\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843082    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-registration-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843082    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-systemd\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843083    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-hostroot\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843117    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-sys\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843130    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843131    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jqb9\" (UniqueName: \"kubernetes.io/projected/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-kube-api-access-9jqb9\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843148    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-hostroot\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843137    4993 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843169    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-hosts-file\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843189    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-etc-selinux\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843203    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-systemd\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843218    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysctl-d\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843230    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/853d1e90-3b6e-4bb5-b6c5-82b1ce141e43-hosts-file\") pod \"node-resolver-pbhv8\" (UID: \"853d1e90-3b6e-4bb5-b6c5-82b1ce141e43\") " pod="openshift-dns/node-resolver-pbhv8"
Nov 03 20:39:58.843242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843239    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-systemd-units\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843265    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-systemd\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843275    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-systemd-units\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843277    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-var-lib-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843279    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-etc-selinux\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843306    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysctl-conf\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843325    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-run\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843330    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-var-lib-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843350    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-device-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843379    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysctl-d\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843379    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-var-lib-kubelet\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843392    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-run\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843413    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-node-log\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843421    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-var-lib-kubelet\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843431    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-host\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843446    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-device-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843463    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9353a680-00bf-4d3a-8437-5a959a40a682-dbus\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843463    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysctl-conf\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843459    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-node-log\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843494    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-host\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843497    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-kubelet\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843536    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9353a680-00bf-4d3a-8437-5a959a40a682-dbus\") pod \"global-pull-secret-syncer-8jhcx\" (UID: \"9353a680-00bf-4d3a-8437-5a959a40a682\") " pod="kube-system/global-pull-secret-syncer-8jhcx"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843571    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-conf-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843593    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-cnibin\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843679    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-conf-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843693    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5e0251d-3ac4-4b42-9175-39ffd668896f-host-slash\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843711    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-cnibin\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843782    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-kubelet\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843815    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-slash\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843836    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5e0251d-3ac4-4b42-9175-39ffd668896f-host-slash\") pod \"iptables-alerter-4dwkb\" (UID: \"e5e0251d-3ac4-4b42-9175-39ffd668896f\") " pod="openshift-network-operator/iptables-alerter-4dwkb"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843842    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-tmp\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843851    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-slash\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843874    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-grpc-tls\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.843879 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843889    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-etc-kubernetes\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843925    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-etc-kubernetes\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843950    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-wtmp\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843965    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e658bf30-634f-4db1-9b41-280a746540a8-host\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.843990    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844004    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-node-exporter-wtmp\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844005    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-cni-netd\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844028    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-run-netns\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844047    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e658bf30-634f-4db1-9b41-280a746540a8-host\") pod \"node-ca-9hqmv\" (UID: \"e658bf30-634f-4db1-9b41-280a746540a8\") " pod="openshift-image-registry/node-ca-9hqmv"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844048    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-system-cni-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844029    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-cni-netd\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844081    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-run-netns\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844096    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-system-cni-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844094    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844117    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-cni-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844138    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844159    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-kubelet\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844177    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-tmp\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844182    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-log-socket\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844208    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-cni-dir\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844216    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-log-socket\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844222    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-socket-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844248    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-thanos-querier-tls\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844260    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-socket-dir\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844217    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-kubelet\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844273    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844298    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-metrics-client-ca\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844320    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-cni-multus\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844319    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-run-ovn-kubernetes\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844343    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844364    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-var-lib-cni-multus\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844367    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-sys\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844392    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844398    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844417    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-etc-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844441    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-root\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844445    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-sys\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844462    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-cnibin\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844484    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-socket-dir-parent\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844486    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-etc-openvswitch\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844508    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-k8s-cni-cncf-io\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844526    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844533    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-multus-certs\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844578    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-multus-socket-dir-parent\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844578    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-multus-certs\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844601    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/737d534b-bee4-44e9-ac5d-124b25de7f72-root\") pod \"node-exporter-6sjbh\" (UID: \"737d534b-bee4-44e9-ac5d-124b25de7f72\") " pod="openshift-monitoring/node-exporter-6sjbh"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844609    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-k8s-cni-cncf-io\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844658    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-cnibin\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844663    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-netns\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844694    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/346aebd9-354c-4683-9eff-58058e9919c7-crio-socket\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844697    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10a65b13-a9e4-4355-8bac-fab245388f16-host-run-netns\") pod \"multus-585md\" (UID: \"10a65b13-a9e4-4355-8bac-fab245388f16\") " pod="openshift-multus/multus-585md"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844720    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844745    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysconfig\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844747    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/346aebd9-354c-4683-9eff-58058e9919c7-crio-socket\") pod \"insights-runtime-extractor-j86dg\" (UID: \"346aebd9-354c-4683-9eff-58058e9919c7\") " pod="openshift-insights/insights-runtime-extractor-j86dg"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844766    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-sys-fs\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844789    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-kubernetes\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844799    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-sysconfig\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844816    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-cni-bin\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844839    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-ovn\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844847    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/32cf1fce-f61e-4870-aff9-a9171e90b09f-sys-fs\") pod \"aws-ebs-csi-driver-node-9xb2h\" (UID: \"32cf1fce-f61e-4870-aff9-a9171e90b09f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9xb2h"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844866    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844888    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-host-cni-bin\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844936    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/11516670-b4cd-43cc-8c41-64e5bfed2b5a-run-ovn\") pod \"ovnkube-node-pwvcf\" (UID: \"11516670-b4cd-43cc-8c41-64e5bfed2b5a\") " pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.844976    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-system-cni-dir\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.845124    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ffcf342-bccb-4362-b79e-8faf3d337b17-system-cni-dir\") pod \"multus-additional-cni-plugins-lxzlf\" (UID: \"0ffcf342-bccb-4362-b79e-8faf3d337b17\") " pod="openshift-multus/multus-additional-cni-plugins-lxzlf"
Nov 03 20:39:58.845599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.845174    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1169f5bf-4ec8-4f85-aba9-857c3a666a3f-etc-kubernetes\") pod \"tuned-dm2zp\" (UID: \"1169f5bf-4ec8-4f85-aba9-857c3a666a3f\") " pod="openshift-cluster-node-tuning-operator/tuned-dm2zp"
Nov 03 20:39:58.848131 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.845814    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.848131 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.846818    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-metrics-client-ca\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.848131 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.846881    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-grpc-tls\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.848131 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.847024    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.848131 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.847475    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.848131 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.847523    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-thanos-querier-tls\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.848131 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.847920    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.904172 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.904147    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jqb9\" (UniqueName: \"kubernetes.io/projected/9ba3faf5-eacc-43e1-9d00-78adbe1ecd00-kube-api-access-9jqb9\") pod \"thanos-querier-d45496948-n4hjf\" (UID: \"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00\") " pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:58.934005 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.933989    4993 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 20:39:58.950462 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:58.950426    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:39:59.370429 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:59.370402    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-d45496948-n4hjf"]
Nov 03 20:39:59.372819 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:39:59.372780    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ba3faf5_eacc_43e1_9d00_78adbe1ecd00.slice/crio-fe5e7b600e51be0a73e48f2c184251aaa2bcdafc44e68b8c6859a6208acf82e3 WatchSource:0}: Error finding container fe5e7b600e51be0a73e48f2c184251aaa2bcdafc44e68b8c6859a6208acf82e3: Status 404 returned error can't find the container with id fe5e7b600e51be0a73e48f2c184251aaa2bcdafc44e68b8c6859a6208acf82e3
Nov 03 20:39:59.623342 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:39:59.623260    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d45496948-n4hjf" event={"ID":"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00","Type":"ContainerStarted","Data":"fe5e7b600e51be0a73e48f2c184251aaa2bcdafc44e68b8c6859a6208acf82e3"}
Nov 03 20:40:00.266865 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:00.266835    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-45x6l"
Nov 03 20:40:00.628781 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:00.628697    4993 generic.go:356] "Generic (PLEG): container finished" podID="737d534b-bee4-44e9-ac5d-124b25de7f72" containerID="5f65f6ce53a27f42f978dc5877762f5aad1917ebfa941c6f13f83407087b6619" exitCode=0
Nov 03 20:40:00.629232 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:00.628778    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sjbh" event={"ID":"737d534b-bee4-44e9-ac5d-124b25de7f72","Type":"ContainerDied","Data":"5f65f6ce53a27f42f978dc5877762f5aad1917ebfa941c6f13f83407087b6619"}
Nov 03 20:40:01.634165 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:01.634133    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sjbh" event={"ID":"737d534b-bee4-44e9-ac5d-124b25de7f72","Type":"ContainerStarted","Data":"2cd55706aedcff63b93f08faf8f385b933ea4162ddec6c14b59ad73bb861fd34"}
Nov 03 20:40:01.634553 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:01.634186    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6sjbh" event={"ID":"737d534b-bee4-44e9-ac5d-124b25de7f72","Type":"ContainerStarted","Data":"8331bf2f3c18bd4ef15886d6dabf7ee0d78e175e191e7d64d5b43a15352b45c8"}
Nov 03 20:40:01.669998 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:40:01.669966    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:40:02.583000 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:02.582968    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:40:03.642969 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:03.642894    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d45496948-n4hjf" event={"ID":"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00","Type":"ContainerStarted","Data":"6fc5a311c42b4a1406580fa563805e2ef90bda8b013b3f791ae839508493a95f"}
Nov 03 20:40:03.643362 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:03.642964    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d45496948-n4hjf" event={"ID":"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00","Type":"ContainerStarted","Data":"dec61270732fb48788768137d55e46a16c2c7c1063d33530d1c7ebfbd7b8299c"}
Nov 03 20:40:03.643362 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:03.643001    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d45496948-n4hjf" event={"ID":"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00","Type":"ContainerStarted","Data":"f4bcb529e4971ba13177d201d54f8082dbc304e0c0df8fdae537e0c189c75f74"}
Nov 03 20:40:05.095063 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:05.095027    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-gwzjt"
Nov 03 20:40:05.254881 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:05.254083    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:40:05.655011 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:05.654937    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d45496948-n4hjf" event={"ID":"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00","Type":"ContainerStarted","Data":"2525e2858d1ef6340465ea103471b72b526d79e0a45214abcb297f63614bc283"}
Nov 03 20:40:05.655011 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:05.654998    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d45496948-n4hjf" event={"ID":"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00","Type":"ContainerStarted","Data":"703b2d6468e7cb54145bc6e6dcf376c6c6893405277dd49da863525dd4bc2235"}
Nov 03 20:40:05.655169 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:05.655024    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-d45496948-n4hjf" event={"ID":"9ba3faf5-eacc-43e1-9d00-78adbe1ecd00","Type":"ContainerStarted","Data":"b17d90db76975a9bf4d332cb4bd238987d6b57e8cb5b279f6e468e2c0557be10"}
Nov 03 20:40:05.655169 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:05.655104    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:40:05.858594 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:05.858535    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-d45496948-n4hjf" podStartSLOduration=3.574051743 podStartE2EDuration="8.858515859s" totalImagesPullingTime="5.284464116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:39:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:40:05.858316575 +0000 UTC m=+8.403022945" watchObservedRunningTime="2026-11-03 20:40:05.858515859 +0000 UTC m=+8.403222218"
Nov 03 20:40:06.512417 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:06.512387    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-h952v"
Nov 03 20:40:06.755680 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:06.755652    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:40:06.755852 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:06.755841    4993 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 03 20:40:06.856333 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:06.856260    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pwvcf"
Nov 03 20:40:07.955095 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:07.955064    4993 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-142-223.us-west-1.compute.internal" event="NodeReady"
Nov 03 20:40:11.666227 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:11.666201    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-d45496948-n4hjf"
Nov 03 20:40:15.977123 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:15.977086    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-858ff676cd-wfcwm"]
Nov 03 20:40:15.980440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:15.980419    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.032965 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.032938    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Nov 03 20:40:16.032965 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.032939    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Nov 03 20:40:16.033156 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.032983    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Nov 03 20:40:16.074932 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.074907    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-trusted-ca-bundle\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.075077 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.074944    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-serving-cert\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.075077 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.074973    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-oauth-config\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.075077 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.075058    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-oauth-serving-cert\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.075252 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.075102    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7sf7\" (UniqueName: \"kubernetes.io/projected/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-kube-api-access-z7sf7\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.075252 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.075165    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-config\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.075252 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.075194    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-tmp\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.075350 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.075279    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-service-ca\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.082757 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.082734    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Nov 03 20:40:16.132319 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.132294    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ltwhn\""
Nov 03 20:40:16.133048 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.133029    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Nov 03 20:40:16.176417 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.176389    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-trusted-ca-bundle\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.176540 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.176428    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-serving-cert\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.176540 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.176451    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-oauth-config\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.176540 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.176479    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-oauth-serving-cert\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.176540 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.176515    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7sf7\" (UniqueName: \"kubernetes.io/projected/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-kube-api-access-z7sf7\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.176781 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.176560    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-config\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.176781 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.176599    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-tmp\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.176781 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.176654    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-service-ca\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.177441 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.177369    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-service-ca\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.177441 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.177370    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-config\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.177613 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.177452    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-oauth-serving-cert\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.177613 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.177580    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-tmp\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.179745 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.179711    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-oauth-config\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.179848 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.179753    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-serving-cert\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.183191 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.183172    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Nov 03 20:40:16.184089 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.184070    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-858ff676cd-wfcwm"]
Nov 03 20:40:16.188180 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.188154    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-trusted-ca-bundle\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.237957 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.237907    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7sf7\" (UniqueName: \"kubernetes.io/projected/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-kube-api-access-z7sf7\") pod \"console-858ff676cd-wfcwm\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") " pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.293472 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.293450    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:16.674340 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.674261    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-858ff676cd-wfcwm"]
Nov 03 20:40:16.678375 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:40:16.678348    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75b1beb9_cfde_409e_9f5f_9e37ef96c78e.slice/crio-991719fa278df06cbecfe151bb9a7eef2d29ccb85ea82a68dd6818598ceec3cb WatchSource:0}: Error finding container 991719fa278df06cbecfe151bb9a7eef2d29ccb85ea82a68dd6818598ceec3cb: Status 404 returned error can't find the container with id 991719fa278df06cbecfe151bb9a7eef2d29ccb85ea82a68dd6818598ceec3cb
Nov 03 20:40:16.692764 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:16.692734    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-858ff676cd-wfcwm" event={"ID":"75b1beb9-cfde-409e-9f5f-9e37ef96c78e","Type":"ContainerStarted","Data":"991719fa278df06cbecfe151bb9a7eef2d29ccb85ea82a68dd6818598ceec3cb"}
Nov 03 20:40:20.708790 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:20.708748    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-858ff676cd-wfcwm" event={"ID":"75b1beb9-cfde-409e-9f5f-9e37ef96c78e","Type":"ContainerStarted","Data":"517d6c0c67ff3247887b135ec20997876c787b318162002219a7aa6aab328f8d"}
Nov 03 20:40:20.826921 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:20.826854    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-858ff676cd-wfcwm" podStartSLOduration=2.456065522 podStartE2EDuration="5.826838487s" totalImagesPullingTime="3.370772965s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:40:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:40:20.826433529 +0000 UTC m=+23.371139894" watchObservedRunningTime="2026-11-03 20:40:20.826838487 +0000 UTC m=+23.371544858"
Nov 03 20:40:23.721899 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:23.721855    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-hd6z4" event={"ID":"42903eab-e4bc-4f97-9e73-1109e9f5e7ab","Type":"ContainerStarted","Data":"01012315292e0fb76ebe2f67a78fb67f482593ef4f5cfea72019fe408c93031a"}
Nov 03 20:40:23.722450 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:23.722420    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-hd6z4"
Nov 03 20:40:23.740764 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:23.740741    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-hd6z4"
Nov 03 20:40:25.258449 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:25.258411    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l9p4t"
Nov 03 20:40:26.294235 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:26.294193    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:26.294691 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:26.294258    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:26.299832 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:26.299806    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:26.737207 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:26.737175    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:40:31.674128 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:40:31.674087    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:40:49.165354 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.165325    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-775d9b5cdd-2g57m"]
Nov 03 20:40:49.200990 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.200963    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.278018 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.277993    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-775d9b5cdd-2g57m"]
Nov 03 20:40:49.323957 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.323929    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/114fe252-b4a3-426f-a27e-884fc1f263b7-tmp\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.324069 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.323977    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/114fe252-b4a3-426f-a27e-884fc1f263b7-service-ca\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.324069 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.324004    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/114fe252-b4a3-426f-a27e-884fc1f263b7-oauth-serving-cert\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.324157 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.324090    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/114fe252-b4a3-426f-a27e-884fc1f263b7-trusted-ca-bundle\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.324200 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.324182    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/114fe252-b4a3-426f-a27e-884fc1f263b7-console-serving-cert\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.324235 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.324208    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/114fe252-b4a3-426f-a27e-884fc1f263b7-console-oauth-config\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.324235 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.324227    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb6tt\" (UniqueName: \"kubernetes.io/projected/114fe252-b4a3-426f-a27e-884fc1f263b7-kube-api-access-xb6tt\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.324327 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.324246    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/114fe252-b4a3-426f-a27e-884fc1f263b7-console-config\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.425219 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.425174    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/114fe252-b4a3-426f-a27e-884fc1f263b7-console-serving-cert\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.425219 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.425201    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/114fe252-b4a3-426f-a27e-884fc1f263b7-console-oauth-config\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.425219 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.425219    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb6tt\" (UniqueName: \"kubernetes.io/projected/114fe252-b4a3-426f-a27e-884fc1f263b7-kube-api-access-xb6tt\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.425417 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.425237    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/114fe252-b4a3-426f-a27e-884fc1f263b7-console-config\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.425417 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.425269    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/114fe252-b4a3-426f-a27e-884fc1f263b7-tmp\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.425417 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.425299    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/114fe252-b4a3-426f-a27e-884fc1f263b7-service-ca\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.425417 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.425317    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/114fe252-b4a3-426f-a27e-884fc1f263b7-oauth-serving-cert\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.425417 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.425334    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/114fe252-b4a3-426f-a27e-884fc1f263b7-trusted-ca-bundle\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.426079 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.426059    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/114fe252-b4a3-426f-a27e-884fc1f263b7-tmp\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.426399 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.426372    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/114fe252-b4a3-426f-a27e-884fc1f263b7-console-config\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.426478 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.426437    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/114fe252-b4a3-426f-a27e-884fc1f263b7-service-ca\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.426651 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.426610    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/114fe252-b4a3-426f-a27e-884fc1f263b7-oauth-serving-cert\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.426936 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.426914    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/114fe252-b4a3-426f-a27e-884fc1f263b7-trusted-ca-bundle\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.427807 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.427785    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/114fe252-b4a3-426f-a27e-884fc1f263b7-console-oauth-config\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.427889 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.427856    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/114fe252-b4a3-426f-a27e-884fc1f263b7-console-serving-cert\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.485980 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.485954    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb6tt\" (UniqueName: \"kubernetes.io/projected/114fe252-b4a3-426f-a27e-884fc1f263b7-kube-api-access-xb6tt\") pod \"console-775d9b5cdd-2g57m\" (UID: \"114fe252-b4a3-426f-a27e-884fc1f263b7\") " pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.525113 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.525086    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:49.927997 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:49.927965    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-775d9b5cdd-2g57m"]
Nov 03 20:40:49.959679 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:40:49.959652    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod114fe252_b4a3_426f_a27e_884fc1f263b7.slice/crio-0ddc3de36b44ad44ee184c119ca234ba5745b5bcd5f68ca966dee8be5d5fe0a7 WatchSource:0}: Error finding container 0ddc3de36b44ad44ee184c119ca234ba5745b5bcd5f68ca966dee8be5d5fe0a7: Status 404 returned error can't find the container with id 0ddc3de36b44ad44ee184c119ca234ba5745b5bcd5f68ca966dee8be5d5fe0a7
Nov 03 20:40:50.815304 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:50.815269    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-775d9b5cdd-2g57m" event={"ID":"114fe252-b4a3-426f-a27e-884fc1f263b7","Type":"ContainerStarted","Data":"ab8cad4519ad9f11196eaaa0bdf26d4074ab75c6bf97015385c495deded24269"}
Nov 03 20:40:50.815695 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:50.815316    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-775d9b5cdd-2g57m" event={"ID":"114fe252-b4a3-426f-a27e-884fc1f263b7","Type":"ContainerStarted","Data":"0ddc3de36b44ad44ee184c119ca234ba5745b5bcd5f68ca966dee8be5d5fe0a7"}
Nov 03 20:40:50.932993 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:50.932933    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-775d9b5cdd-2g57m" podStartSLOduration=1.932916434 podStartE2EDuration="1.932916434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:40:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:40:50.932273976 +0000 UTC m=+53.476980380" watchObservedRunningTime="2026-11-03 20:40:50.932916434 +0000 UTC m=+53.477622806"
Nov 03 20:40:51.565644 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.565595    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 03 20:40:51.596305 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.596268    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.649883 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.649859    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Nov 03 20:40:51.650058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.649892    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Nov 03 20:40:51.650058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.650004    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Nov 03 20:40:51.744573 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.744544    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.744773 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.744595    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/14913c4a-3449-4529-81fd-eb1a64fbaa95-config-out\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.744773 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.744610    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14913c4a-3449-4529-81fd-eb1a64fbaa95-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.744773 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.744703    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.744773 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.744743    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxvnw\" (UniqueName: \"kubernetes.io/projected/14913c4a-3449-4529-81fd-eb1a64fbaa95-kube-api-access-nxvnw\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.744970 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.744789    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.744970 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.744815    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-web-config\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.744970 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.744857    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-config-volume\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.744970 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.744882    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/14913c4a-3449-4529-81fd-eb1a64fbaa95-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.744970 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.744916    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.745164 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.745004    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.745164 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.745027    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/14913c4a-3449-4529-81fd-eb1a64fbaa95-tls-assets\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.745164 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.745044    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14913c4a-3449-4529-81fd-eb1a64fbaa95-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.749950 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.749929    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Nov 03 20:40:51.750064 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.750008    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Nov 03 20:40:51.750988 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.750972    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 03 20:40:51.799036 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.799015    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Nov 03 20:40:51.799036 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.799030    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Nov 03 20:40:51.799223 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.799039    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Nov 03 20:40:51.799223 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.799120    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Nov 03 20:40:51.799223 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.799155    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-65c9p\""
Nov 03 20:40:51.845483 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.845427    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.845483 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.845463    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-web-config\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.845483 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.845483    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-config-volume\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.845993 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.845501    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/14913c4a-3449-4529-81fd-eb1a64fbaa95-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.845993 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.845520    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.845993 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.845548    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.845993 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.845571    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/14913c4a-3449-4529-81fd-eb1a64fbaa95-tls-assets\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.845993 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.845597    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14913c4a-3449-4529-81fd-eb1a64fbaa95-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.845993 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.845622    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.845993 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.845708    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/14913c4a-3449-4529-81fd-eb1a64fbaa95-config-out\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.845993 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.845731    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14913c4a-3449-4529-81fd-eb1a64fbaa95-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.845993 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.845760    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.845993 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.845786    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxvnw\" (UniqueName: \"kubernetes.io/projected/14913c4a-3449-4529-81fd-eb1a64fbaa95-kube-api-access-nxvnw\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.845993 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.845949    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/14913c4a-3449-4529-81fd-eb1a64fbaa95-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.847656 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.847606    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14913c4a-3449-4529-81fd-eb1a64fbaa95-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.848008 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.847929    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/14913c4a-3449-4529-81fd-eb1a64fbaa95-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.848814 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.848780    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/14913c4a-3449-4529-81fd-eb1a64fbaa95-config-out\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.849162 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.849139    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.849375 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.849353    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-config-volume\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.849471 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.849385    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.849651 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.849612    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.849775 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.849660    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-web-config\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.850259 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.850236    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/14913c4a-3449-4529-81fd-eb1a64fbaa95-tls-assets\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.850447 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.850425    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.850893 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.850870    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/14913c4a-3449-4529-81fd-eb1a64fbaa95-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:51.906708 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:51.906680    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxvnw\" (UniqueName: \"kubernetes.io/projected/14913c4a-3449-4529-81fd-eb1a64fbaa95-kube-api-access-nxvnw\") pod \"alertmanager-main-0\" (UID: \"14913c4a-3449-4529-81fd-eb1a64fbaa95\") " pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:52.003871 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:52.003848    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Nov 03 20:40:52.505579 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:52.505552    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Nov 03 20:40:52.508283 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:40:52.508258    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14913c4a_3449_4529_81fd_eb1a64fbaa95.slice/crio-a9001ad51748e07837f01366f1127be31421206bb665f82a823d61d4c20dbf64 WatchSource:0}: Error finding container a9001ad51748e07837f01366f1127be31421206bb665f82a823d61d4c20dbf64: Status 404 returned error can't find the container with id a9001ad51748e07837f01366f1127be31421206bb665f82a823d61d4c20dbf64
Nov 03 20:40:52.822599 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:52.822524    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14913c4a-3449-4529-81fd-eb1a64fbaa95","Type":"ContainerStarted","Data":"a9001ad51748e07837f01366f1127be31421206bb665f82a823d61d4c20dbf64"}
Nov 03 20:40:54.834218 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:54.834183    4993 generic.go:356] "Generic (PLEG): container finished" podID="14913c4a-3449-4529-81fd-eb1a64fbaa95" containerID="ef9809d7fc85496a7ddd0eacd26a8a7755234da65d8241456afcc002670db5e1" exitCode=0
Nov 03 20:40:54.834667 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:54.834230    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14913c4a-3449-4529-81fd-eb1a64fbaa95","Type":"ContainerDied","Data":"ef9809d7fc85496a7ddd0eacd26a8a7755234da65d8241456afcc002670db5e1"}
Nov 03 20:40:57.847071 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:57.847039    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14913c4a-3449-4529-81fd-eb1a64fbaa95","Type":"ContainerStarted","Data":"702997a96ff2549d6c3562a5bfadab0798d930b44041cdf4515597412343750a"}
Nov 03 20:40:57.847424 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:57.847122    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14913c4a-3449-4529-81fd-eb1a64fbaa95","Type":"ContainerStarted","Data":"229fc53eba4ffa81aafb96d88577895d319a64f024aa167aef786ea0e7ef55e2"}
Nov 03 20:40:57.847424 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:57.847147    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14913c4a-3449-4529-81fd-eb1a64fbaa95","Type":"ContainerStarted","Data":"b0b2392abec407d67620e8f2ccc855ec958639d80a15c8e278d8af5d8a860282"}
Nov 03 20:40:57.847424 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:57.847173    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14913c4a-3449-4529-81fd-eb1a64fbaa95","Type":"ContainerStarted","Data":"03352c6a4d5dfb2c66f75cb59619f39f2b31bb9b17f26d2fab831bb698cce921"}
Nov 03 20:40:57.847424 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:57.847201    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14913c4a-3449-4529-81fd-eb1a64fbaa95","Type":"ContainerStarted","Data":"c21aa24481dde27062dcfc2773e3f3d86e325472b766f6f289c62c4648cccf64"}
Nov 03 20:40:57.847424 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:57.847227    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"14913c4a-3449-4529-81fd-eb1a64fbaa95","Type":"ContainerStarted","Data":"4b3506b619271e379fddca63f60591a24a7a7f2bc06e3d6f45fd95b7fff838f5"}
Nov 03 20:40:58.123431 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:58.123236    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.196882542 podStartE2EDuration="7.123222595s" totalImagesPullingTime="3.913698491s" totalInitContainerRuntime="12.641562ms" isStatefulPod=false podCreationTimestamp="2026-11-03 20:40:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:40:58.098708263 +0000 UTC m=+60.643414634" watchObservedRunningTime="2026-11-03 20:40:58.123222595 +0000 UTC m=+60.667928959"
Nov 03 20:40:59.525442 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:59.525408    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:59.525899 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:59.525559    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:59.530148 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:59.530127    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:40:59.858187 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:40:59.858103    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-775d9b5cdd-2g57m"
Nov 03 20:41:00.008967 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:00.008935    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-858ff676cd-wfcwm"]
Nov 03 20:41:01.669823 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:41:01.669788    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:41:25.031104 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.031068    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-858ff676cd-wfcwm" podUID="75b1beb9-cfde-409e-9f5f-9e37ef96c78e" containerName="console" containerID="cri-o://517d6c0c67ff3247887b135ec20997876c787b318162002219a7aa6aab328f8d" gracePeriod=15
Nov 03 20:41:25.275288 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.275266    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:41:25.384156 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.384092    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-oauth-serving-cert\") pod \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") "
Nov 03 20:41:25.384156 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.384139    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-tmp\" (UniqueName: \"kubernetes.io/empty-dir/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-tmp\") pod \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") "
Nov 03 20:41:25.384332 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.384186    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-oauth-config\") pod \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") "
Nov 03 20:41:25.384332 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.384207    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-config\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-config\") pod \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") "
Nov 03 20:41:25.384332 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.384234    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-service-ca\") pod \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") "
Nov 03 20:41:25.384332 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.384277    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-trusted-ca-bundle\") pod \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") "
Nov 03 20:41:25.384332 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.384309    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-serving-cert\") pod \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") "
Nov 03 20:41:25.384570 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.384339    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-kube-api-access-z7sf7\" (UniqueName: \"kubernetes.io/projected/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-kube-api-access-z7sf7\") pod \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\" (UID: \"75b1beb9-cfde-409e-9f5f-9e37ef96c78e\") "
Nov 03 20:41:25.384684 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.384528    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-oauth-serving-cert" pod "75b1beb9-cfde-409e-9f5f-9e37ef96c78e" (UID: "75b1beb9-cfde-409e-9f5f-9e37ef96c78e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:41:25.384764 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.384658    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-tmp" pod "75b1beb9-cfde-409e-9f5f-9e37ef96c78e" (UID: "75b1beb9-cfde-409e-9f5f-9e37ef96c78e"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:41:25.384810 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.384768    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-config" pod "75b1beb9-cfde-409e-9f5f-9e37ef96c78e" (UID: "75b1beb9-cfde-409e-9f5f-9e37ef96c78e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:41:25.384810 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.384781    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-trusted-ca-bundle" pod "75b1beb9-cfde-409e-9f5f-9e37ef96c78e" (UID: "75b1beb9-cfde-409e-9f5f-9e37ef96c78e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:41:25.385027 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.384996    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-service-ca" pod "75b1beb9-cfde-409e-9f5f-9e37ef96c78e" (UID: "75b1beb9-cfde-409e-9f5f-9e37ef96c78e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:41:25.386573 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.386540    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-oauth-config" pod "75b1beb9-cfde-409e-9f5f-9e37ef96c78e" (UID: "75b1beb9-cfde-409e-9f5f-9e37ef96c78e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:41:25.386697 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.386610    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-serving-cert" pod "75b1beb9-cfde-409e-9f5f-9e37ef96c78e" (UID: "75b1beb9-cfde-409e-9f5f-9e37ef96c78e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:41:25.386697 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.386653    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-kube-api-access-z7sf7" pod "75b1beb9-cfde-409e-9f5f-9e37ef96c78e" (UID: "75b1beb9-cfde-409e-9f5f-9e37ef96c78e"). InnerVolumeSpecName "kube-api-access-z7sf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:41:25.485467 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.485446    4993 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-trusted-ca-bundle\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:41:25.485467 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.485468    4993 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-serving-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:41:25.485620 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.485477    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7sf7\" (UniqueName: \"kubernetes.io/projected/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-kube-api-access-z7sf7\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:41:25.485620 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.485487    4993 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-oauth-serving-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:41:25.485620 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.485497    4993 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-tmp\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:41:25.485620 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.485505    4993 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-oauth-config\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:41:25.485620 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.485514    4993 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-console-config\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:41:25.485620 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.485525    4993 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/75b1beb9-cfde-409e-9f5f-9e37ef96c78e-service-ca\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:41:25.948689 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.948654    4993 generic.go:356] "Generic (PLEG): container finished" podID="75b1beb9-cfde-409e-9f5f-9e37ef96c78e" containerID="517d6c0c67ff3247887b135ec20997876c787b318162002219a7aa6aab328f8d" exitCode=0
Nov 03 20:41:25.948850 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.948701    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-858ff676cd-wfcwm"
Nov 03 20:41:25.948850 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.948728    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-858ff676cd-wfcwm" event={"ID":"75b1beb9-cfde-409e-9f5f-9e37ef96c78e","Type":"ContainerDied","Data":"517d6c0c67ff3247887b135ec20997876c787b318162002219a7aa6aab328f8d"}
Nov 03 20:41:25.948850 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.948765    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-858ff676cd-wfcwm" event={"ID":"75b1beb9-cfde-409e-9f5f-9e37ef96c78e","Type":"ContainerDied","Data":"991719fa278df06cbecfe151bb9a7eef2d29ccb85ea82a68dd6818598ceec3cb"}
Nov 03 20:41:25.948850 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.948794    4993 scope.go:122] "RemoveContainer" containerID="517d6c0c67ff3247887b135ec20997876c787b318162002219a7aa6aab328f8d"
Nov 03 20:41:25.956845 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.956825    4993 scope.go:122] "RemoveContainer" containerID="517d6c0c67ff3247887b135ec20997876c787b318162002219a7aa6aab328f8d"
Nov 03 20:41:25.957099 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:41:25.957080    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"517d6c0c67ff3247887b135ec20997876c787b318162002219a7aa6aab328f8d\": container with ID starting with 517d6c0c67ff3247887b135ec20997876c787b318162002219a7aa6aab328f8d not found: ID does not exist" containerID="517d6c0c67ff3247887b135ec20997876c787b318162002219a7aa6aab328f8d"
Nov 03 20:41:25.957169 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:25.957111    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"517d6c0c67ff3247887b135ec20997876c787b318162002219a7aa6aab328f8d"} err="failed to get container status \"517d6c0c67ff3247887b135ec20997876c787b318162002219a7aa6aab328f8d\": rpc error: code = NotFound desc = could not find container \"517d6c0c67ff3247887b135ec20997876c787b318162002219a7aa6aab328f8d\": container with ID starting with 517d6c0c67ff3247887b135ec20997876c787b318162002219a7aa6aab328f8d not found: ID does not exist"
Nov 03 20:41:26.112042 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:26.112015    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-858ff676cd-wfcwm"]
Nov 03 20:41:26.161909 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:26.161884    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-858ff676cd-wfcwm"]
Nov 03 20:41:27.574734 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:41:27.574697    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75b1beb9-cfde-409e-9f5f-9e37ef96c78e" path="/var/lib/kubelet/pods/75b1beb9-cfde-409e-9f5f-9e37ef96c78e/volumes"
Nov 03 20:41:31.669569 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:41:31.669533    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:42:01.668337 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:42:01.668299    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:42:31.670039 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:42:31.669999    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:43:01.668335 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:43:01.668293    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:43:18.531557 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.531525    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"]
Nov 03 20:43:18.531980 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.531835    4993 cpu_manager.go:414] "containerMap: removing container" podUID="75b1beb9-cfde-409e-9f5f-9e37ef96c78e" containerName="console"
Nov 03 20:43:18.531980 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.531847    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75b1beb9-cfde-409e-9f5f-9e37ef96c78e" containerName="console"
Nov 03 20:43:18.531980 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.531897    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="75b1beb9-cfde-409e-9f5f-9e37ef96c78e" containerName="console"
Nov 03 20:43:18.533774 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.533752    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"
Nov 03 20:43:18.555336 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.555307    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7jmn\" (UniqueName: \"kubernetes.io/projected/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-kube-api-access-c7jmn\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7\" (UID: \"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"
Nov 03 20:43:18.555453 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.555353    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7\" (UID: \"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"
Nov 03 20:43:18.555498 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.555467    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7\" (UID: \"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"
Nov 03 20:43:18.644583 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.644550    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 03 20:43:18.644583 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.644570    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-dl28n\""
Nov 03 20:43:18.644777 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.644656    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 03 20:43:18.645333 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.645313    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"]
Nov 03 20:43:18.655960 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.655937    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7jmn\" (UniqueName: \"kubernetes.io/projected/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-kube-api-access-c7jmn\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7\" (UID: \"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"
Nov 03 20:43:18.656045 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.655967    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7\" (UID: \"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"
Nov 03 20:43:18.656045 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.655992    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7\" (UID: \"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"
Nov 03 20:43:18.656280 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.656264    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7\" (UID: \"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"
Nov 03 20:43:18.656338 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.656310    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7\" (UID: \"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"
Nov 03 20:43:18.717115 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.717080    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7jmn\" (UniqueName: \"kubernetes.io/projected/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-kube-api-access-c7jmn\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7\" (UID: \"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"
Nov 03 20:43:18.845802 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.845729    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"
Nov 03 20:43:18.956523 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.956495    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"]
Nov 03 20:43:18.958765 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:18.958748    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.030542 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.030510    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 03 20:43:19.030542 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.030515    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 03 20:43:19.030759 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.030590    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 03 20:43:19.030759 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.030604    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 03 20:43:19.030759 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.030665    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 03 20:43:19.030759 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.030700    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 03 20:43:19.030759 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.030719    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cqrrn\""
Nov 03 20:43:19.037875 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.037852    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"]
Nov 03 20:43:19.059933 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.059914    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.060029 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.059942    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.060029 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.059962    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.060029 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.059984    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.060129 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.060086    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.060178 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.060161    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.060212 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.060191    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xdpg\" (UniqueName: \"kubernetes.io/projected/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-kube-api-access-6xdpg\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.160840 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.160777    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.160840 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.160821    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.160996 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.160859    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.160996 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.160878    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xdpg\" (UniqueName: \"kubernetes.io/projected/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-kube-api-access-6xdpg\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.160996 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.160908    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.160996 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.160939    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.160996 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.160974    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.161475 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.161444    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.168525 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.163780    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.168525 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.163875    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.168525 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.163947    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.169046 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.169029    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.214800 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.214778    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"]
Nov 03 20:43:19.217596 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:43:19.217576    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbf585f4_7a81_4a61_bf43_c2b4c2029f1a.slice/crio-c68d23a364bf4ebba4feb29d8f62ff01d56ea449d75974dae000720e55c13087 WatchSource:0}: Error finding container c68d23a364bf4ebba4feb29d8f62ff01d56ea449d75974dae000720e55c13087: Status 404 returned error can't find the container with id c68d23a364bf4ebba4feb29d8f62ff01d56ea449d75974dae000720e55c13087
Nov 03 20:43:19.221332 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.221314    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.221640 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.221606    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xdpg\" (UniqueName: \"kubernetes.io/projected/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-kube-api-access-6xdpg\") pod \"istiod-openshift-gateway-7876bfcb74-2rjp4\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.269065 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.269048    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:19.326993 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.326955    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7" event={"ID":"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a","Type":"ContainerStarted","Data":"c68d23a364bf4ebba4feb29d8f62ff01d56ea449d75974dae000720e55c13087"}
Nov 03 20:43:19.686188 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:19.686154    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"]
Nov 03 20:43:19.689661 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:43:19.689611    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fd09aa6_d1f6_4f41_8eab_fac00f97938e.slice/crio-013d7461f35404bd2a24a46d5823073923c303451f1f24b07e35c3a0b58e9539 WatchSource:0}: Error finding container 013d7461f35404bd2a24a46d5823073923c303451f1f24b07e35c3a0b58e9539: Status 404 returned error can't find the container with id 013d7461f35404bd2a24a46d5823073923c303451f1f24b07e35c3a0b58e9539
Nov 03 20:43:20.332976 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:20.332936    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4" event={"ID":"7fd09aa6-d1f6-4f41-8eab-fac00f97938e","Type":"ContainerStarted","Data":"013d7461f35404bd2a24a46d5823073923c303451f1f24b07e35c3a0b58e9539"}
Nov 03 20:43:25.438375 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:25.438345    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:43:25.438799 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:25.438412    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:43:26.358382 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:26.358350    4993 generic.go:356] "Generic (PLEG): container finished" podID="dbf585f4-7a81-4a61-bf43-c2b4c2029f1a" containerID="b1a6b0d2aba6a8ab9ec72486e5fbc6142df0b50cb47ec744b79d1c22f1fb50bf" exitCode=0
Nov 03 20:43:26.358578 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:26.358429    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7" event={"ID":"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a","Type":"ContainerDied","Data":"b1a6b0d2aba6a8ab9ec72486e5fbc6142df0b50cb47ec744b79d1c22f1fb50bf"}
Nov 03 20:43:26.360154 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:26.360134    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4" event={"ID":"7fd09aa6-d1f6-4f41-8eab-fac00f97938e","Type":"ContainerStarted","Data":"a2e2644ba8e1e51bc8061f3ab46a483c81d5706668cce9c81da60bbd0061814d"}
Nov 03 20:43:26.360354 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:26.360339    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:26.361979 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:26.361954    4993 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-2rjp4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 20:43:26.362067 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:26.361994    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4" podUID="7fd09aa6-d1f6-4f41-8eab-fac00f97938e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 20:43:26.596479 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:26.596429    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4" podStartSLOduration=2.850636401 podStartE2EDuration="8.596414898s" totalImagesPullingTime="5.745778497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:43:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:43:26.596011506 +0000 UTC m=+209.140717876" watchObservedRunningTime="2026-11-03 20:43:26.596414898 +0000 UTC m=+209.141121268"
Nov 03 20:43:27.364821 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:27.364792    4993 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-2rjp4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 20:43:27.364982 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:27.364843    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4" podUID="7fd09aa6-d1f6-4f41-8eab-fac00f97938e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 20:43:30.365334 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:30.365302    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:43:30.377221 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:30.377188    4993 generic.go:356] "Generic (PLEG): container finished" podID="dbf585f4-7a81-4a61-bf43-c2b4c2029f1a" containerID="0492c97add92f0c3e90f418f6f7aac58644daa80536bdebf52de948fc29cd245" exitCode=0
Nov 03 20:43:30.377350 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:30.377276    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7" event={"ID":"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a","Type":"ContainerDied","Data":"0492c97add92f0c3e90f418f6f7aac58644daa80536bdebf52de948fc29cd245"}
Nov 03 20:43:31.676678 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:43:31.676622    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:43:37.408275 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:37.408240    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7" event={"ID":"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a","Type":"ContainerStarted","Data":"0e586fa062f70199fa0318eb1805e0156993038fa2eb348ff601546ee2d1bd33"}
Nov 03 20:43:37.525651 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:37.525588    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7" podStartSLOduration=2.468095012 podStartE2EDuration="19.525574808s" totalImagesPullingTime="17.057479796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:43:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:43:37.524998292 +0000 UTC m=+220.069704663" watchObservedRunningTime="2026-11-03 20:43:37.525574808 +0000 UTC m=+220.070281177"
Nov 03 20:43:38.414024 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:38.413988    4993 generic.go:356] "Generic (PLEG): container finished" podID="dbf585f4-7a81-4a61-bf43-c2b4c2029f1a" containerID="0e586fa062f70199fa0318eb1805e0156993038fa2eb348ff601546ee2d1bd33" exitCode=0
Nov 03 20:43:38.414470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:38.414054    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7" event={"ID":"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a","Type":"ContainerDied","Data":"0e586fa062f70199fa0318eb1805e0156993038fa2eb348ff601546ee2d1bd33"}
Nov 03 20:43:39.541728 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:39.541704    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"
Nov 03 20:43:39.636994 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:39.636969    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-bundle\" (UniqueName: \"kubernetes.io/empty-dir/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-bundle\") pod \"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a\" (UID: \"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a\") "
Nov 03 20:43:39.637125 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:39.637030    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-kube-api-access-c7jmn\" (UniqueName: \"kubernetes.io/projected/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-kube-api-access-c7jmn\") pod \"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a\" (UID: \"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a\") "
Nov 03 20:43:39.637125 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:39.637065    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-util\" (UniqueName: \"kubernetes.io/empty-dir/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-util\") pod \"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a\" (UID: \"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a\") "
Nov 03 20:43:39.637406 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:39.637388    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-bundle" pod "dbf585f4-7a81-4a61-bf43-c2b4c2029f1a" (UID: "dbf585f4-7a81-4a61-bf43-c2b4c2029f1a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:43:39.639374 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:39.639346    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-kube-api-access-c7jmn" pod "dbf585f4-7a81-4a61-bf43-c2b4c2029f1a" (UID: "dbf585f4-7a81-4a61-bf43-c2b4c2029f1a"). InnerVolumeSpecName "kube-api-access-c7jmn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:43:39.641615 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:39.641595    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-util" pod "dbf585f4-7a81-4a61-bf43-c2b4c2029f1a" (UID: "dbf585f4-7a81-4a61-bf43-c2b4c2029f1a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:43:39.738047 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:39.738014    4993 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-bundle\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:43:39.738047 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:39.738046    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7jmn\" (UniqueName: \"kubernetes.io/projected/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-kube-api-access-c7jmn\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:43:39.738047 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:39.738056    4993 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dbf585f4-7a81-4a61-bf43-c2b4c2029f1a-util\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:43:40.422404 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:40.422368    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7" event={"ID":"dbf585f4-7a81-4a61-bf43-c2b4c2029f1a","Type":"ContainerDied","Data":"c68d23a364bf4ebba4feb29d8f62ff01d56ea449d75974dae000720e55c13087"}
Nov 03 20:43:40.422567 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:40.422420    4993 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c68d23a364bf4ebba4feb29d8f62ff01d56ea449d75974dae000720e55c13087"
Nov 03 20:43:40.422567 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:43:40.422378    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644z4pn7"
Nov 03 20:44:01.669980 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:44:01.669942    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:44:14.487166 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.487134    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"]
Nov 03 20:44:14.487693 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.487426    4993 cpu_manager.go:414] "containerMap: removing container" podUID="dbf585f4-7a81-4a61-bf43-c2b4c2029f1a" containerName="util"
Nov 03 20:44:14.487693 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.487438    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbf585f4-7a81-4a61-bf43-c2b4c2029f1a" containerName="util"
Nov 03 20:44:14.487693 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.487457    4993 cpu_manager.go:414] "containerMap: removing container" podUID="dbf585f4-7a81-4a61-bf43-c2b4c2029f1a" containerName="extract"
Nov 03 20:44:14.487693 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.487462    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbf585f4-7a81-4a61-bf43-c2b4c2029f1a" containerName="extract"
Nov 03 20:44:14.487693 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.487478    4993 cpu_manager.go:414] "containerMap: removing container" podUID="dbf585f4-7a81-4a61-bf43-c2b4c2029f1a" containerName="pull"
Nov 03 20:44:14.487693 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.487482    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbf585f4-7a81-4a61-bf43-c2b4c2029f1a" containerName="pull"
Nov 03 20:44:14.487693 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.487542    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="dbf585f4-7a81-4a61-bf43-c2b4c2029f1a" containerName="extract"
Nov 03 20:44:14.489367 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.489353    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.601206 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.601172    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 03 20:44:14.601403 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.601178    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 03 20:44:14.601477 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.601204    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 03 20:44:14.601477 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.601236    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 03 20:44:14.601477 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.601239    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 03 20:44:14.601477 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.601244    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e3df644-2f23-4f39-9c87-434b68244aba-metrics-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.601620 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.601526    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8e3df644-2f23-4f39-9c87-434b68244aba-manifests\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.601620 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.601263    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-7q7qx\""
Nov 03 20:44:14.601620 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.601553    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8e3df644-2f23-4f39-9c87-434b68244aba-webhook-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.601747 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.601660    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e3df644-2f23-4f39-9c87-434b68244aba-tmp\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.601747 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.601688    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8e3df644-2f23-4f39-9c87-434b68244aba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.601819 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.601740    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wjh7\" (UniqueName: \"kubernetes.io/projected/8e3df644-2f23-4f39-9c87-434b68244aba-kube-api-access-7wjh7\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.602057 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.602039    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"]
Nov 03 20:44:14.702979 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.702949    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e3df644-2f23-4f39-9c87-434b68244aba-tmp\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.702979 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.702979    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8e3df644-2f23-4f39-9c87-434b68244aba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.703202 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.703009    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wjh7\" (UniqueName: \"kubernetes.io/projected/8e3df644-2f23-4f39-9c87-434b68244aba-kube-api-access-7wjh7\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.703202 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.703054    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e3df644-2f23-4f39-9c87-434b68244aba-metrics-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.703202 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.703073    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8e3df644-2f23-4f39-9c87-434b68244aba-manifests\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.703202 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.703087    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8e3df644-2f23-4f39-9c87-434b68244aba-webhook-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.703473 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.703451    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8e3df644-2f23-4f39-9c87-434b68244aba-manifests\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.705578 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.705544    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8e3df644-2f23-4f39-9c87-434b68244aba-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.705709 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.705578    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e3df644-2f23-4f39-9c87-434b68244aba-metrics-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.705829 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.705810    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e3df644-2f23-4f39-9c87-434b68244aba-tmp\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.705940 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.705924    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8e3df644-2f23-4f39-9c87-434b68244aba-webhook-cert\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.766615 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.766545    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wjh7\" (UniqueName: \"kubernetes.io/projected/8e3df644-2f23-4f39-9c87-434b68244aba-kube-api-access-7wjh7\") pod \"opendatahub-operator-controller-manager-78789f7d7-xvq57\" (UID: \"8e3df644-2f23-4f39-9c87-434b68244aba\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:14.802013 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:14.801981    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:15.305494 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:15.305455    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"]
Nov 03 20:44:15.308544 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:44:15.308515    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e3df644_2f23_4f39_9c87_434b68244aba.slice/crio-7205937ff3fca74b6b4aa6b57af2da3103da3d104bd5f2b66b0cd375bade000c WatchSource:0}: Error finding container 7205937ff3fca74b6b4aa6b57af2da3103da3d104bd5f2b66b0cd375bade000c: Status 404 returned error can't find the container with id 7205937ff3fca74b6b4aa6b57af2da3103da3d104bd5f2b66b0cd375bade000c
Nov 03 20:44:15.544348 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:15.544307    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57" event={"ID":"8e3df644-2f23-4f39-9c87-434b68244aba","Type":"ContainerStarted","Data":"7205937ff3fca74b6b4aa6b57af2da3103da3d104bd5f2b66b0cd375bade000c"}
Nov 03 20:44:17.142961 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.142928    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"]
Nov 03 20:44:17.146010 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.145349    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"
Nov 03 20:44:17.200757 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.200605    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 03 20:44:17.200757 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.200705    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 03 20:44:17.223573 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.223515    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qtv6\" (UniqueName: \"kubernetes.io/projected/b53b8830-8d79-41aa-a8a3-29bad57a7100-kube-api-access-8qtv6\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh\" (UID: \"b53b8830-8d79-41aa-a8a3-29bad57a7100\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"
Nov 03 20:44:17.223782 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.223573    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b53b8830-8d79-41aa-a8a3-29bad57a7100-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh\" (UID: \"b53b8830-8d79-41aa-a8a3-29bad57a7100\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"
Nov 03 20:44:17.223782 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.223615    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b53b8830-8d79-41aa-a8a3-29bad57a7100-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh\" (UID: \"b53b8830-8d79-41aa-a8a3-29bad57a7100\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"
Nov 03 20:44:17.250729 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.250696    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-bdsdc\""
Nov 03 20:44:17.251968 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.251939    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"]
Nov 03 20:44:17.324741 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.324705    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b53b8830-8d79-41aa-a8a3-29bad57a7100-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh\" (UID: \"b53b8830-8d79-41aa-a8a3-29bad57a7100\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"
Nov 03 20:44:17.324924 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.324829    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qtv6\" (UniqueName: \"kubernetes.io/projected/b53b8830-8d79-41aa-a8a3-29bad57a7100-kube-api-access-8qtv6\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh\" (UID: \"b53b8830-8d79-41aa-a8a3-29bad57a7100\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"
Nov 03 20:44:17.324924 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.324859    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b53b8830-8d79-41aa-a8a3-29bad57a7100-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh\" (UID: \"b53b8830-8d79-41aa-a8a3-29bad57a7100\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"
Nov 03 20:44:17.325176 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.325150    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b53b8830-8d79-41aa-a8a3-29bad57a7100-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh\" (UID: \"b53b8830-8d79-41aa-a8a3-29bad57a7100\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"
Nov 03 20:44:17.325307 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.325185    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b53b8830-8d79-41aa-a8a3-29bad57a7100-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh\" (UID: \"b53b8830-8d79-41aa-a8a3-29bad57a7100\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"
Nov 03 20:44:17.405982 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.405947    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qtv6\" (UniqueName: \"kubernetes.io/projected/b53b8830-8d79-41aa-a8a3-29bad57a7100-kube-api-access-8qtv6\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh\" (UID: \"b53b8830-8d79-41aa-a8a3-29bad57a7100\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"
Nov 03 20:44:17.461828 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.461796    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"
Nov 03 20:44:17.849386 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:17.849309    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"]
Nov 03 20:44:17.882641 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:44:17.882594    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb53b8830_8d79_41aa_a8a3_29bad57a7100.slice/crio-bca885cd4287f5379543d38199e11ac2c817b8d47caa3ad8358f4d25996f840f WatchSource:0}: Error finding container bca885cd4287f5379543d38199e11ac2c817b8d47caa3ad8358f4d25996f840f: Status 404 returned error can't find the container with id bca885cd4287f5379543d38199e11ac2c817b8d47caa3ad8358f4d25996f840f
Nov 03 20:44:18.559280 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:18.559244    4993 generic.go:356] "Generic (PLEG): container finished" podID="b53b8830-8d79-41aa-a8a3-29bad57a7100" containerID="b7fc504bc08f51381e90ec641fddd7d7c2f387d904cb89a4fe61f067c47e8c15" exitCode=0
Nov 03 20:44:18.559758 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:18.559337    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh" event={"ID":"b53b8830-8d79-41aa-a8a3-29bad57a7100","Type":"ContainerDied","Data":"b7fc504bc08f51381e90ec641fddd7d7c2f387d904cb89a4fe61f067c47e8c15"}
Nov 03 20:44:18.559758 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:18.559385    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh" event={"ID":"b53b8830-8d79-41aa-a8a3-29bad57a7100","Type":"ContainerStarted","Data":"bca885cd4287f5379543d38199e11ac2c817b8d47caa3ad8358f4d25996f840f"}
Nov 03 20:44:19.564895 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:19.564799    4993 generic.go:356] "Generic (PLEG): container finished" podID="8e3df644-2f23-4f39-9c87-434b68244aba" containerID="028f5f4452395675f6de6d22dfdc3a4136ada1a72474b6f9c8af34f32c21c845" exitCode=0
Nov 03 20:44:19.564895 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:19.564869    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57" event={"ID":"8e3df644-2f23-4f39-9c87-434b68244aba","Type":"ContainerDied","Data":"028f5f4452395675f6de6d22dfdc3a4136ada1a72474b6f9c8af34f32c21c845"}
Nov 03 20:44:20.575312 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:20.575275    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57" event={"ID":"8e3df644-2f23-4f39-9c87-434b68244aba","Type":"ContainerStarted","Data":"c21801fdb27d1fe69adf6f523c5a93a312c4a647775a98b40e276b95de588b86"}
Nov 03 20:44:20.575820 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:20.575406    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:20.577233 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:20.577210    4993 generic.go:356] "Generic (PLEG): container finished" podID="b53b8830-8d79-41aa-a8a3-29bad57a7100" containerID="5b3172b4a8b7f3a6f9d052873bc5134f3172c2bcc99a56c3d06bd43d59285a7f" exitCode=0
Nov 03 20:44:20.577359 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:20.577279    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh" event={"ID":"b53b8830-8d79-41aa-a8a3-29bad57a7100","Type":"ContainerDied","Data":"5b3172b4a8b7f3a6f9d052873bc5134f3172c2bcc99a56c3d06bd43d59285a7f"}
Nov 03 20:44:20.752832 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:20.752778    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57" podStartSLOduration=2.075559001 podStartE2EDuration="6.752765523s" totalImagesPullingTime="4.235147734s" totalInitContainerRuntime="442.058788ms" isStatefulPod=false podCreationTimestamp="2026-11-03 20:44:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:44:20.752338698 +0000 UTC m=+263.297045079" watchObservedRunningTime="2026-11-03 20:44:20.752765523 +0000 UTC m=+263.297471893"
Nov 03 20:44:21.582806 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:21.582728    4993 generic.go:356] "Generic (PLEG): container finished" podID="b53b8830-8d79-41aa-a8a3-29bad57a7100" containerID="cb57652776989a603aa9ea73650bae5f6ee88fd11351d76d89beaaa2ce8d332b" exitCode=0
Nov 03 20:44:21.583155 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:21.582806    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh" event={"ID":"b53b8830-8d79-41aa-a8a3-29bad57a7100","Type":"ContainerDied","Data":"cb57652776989a603aa9ea73650bae5f6ee88fd11351d76d89beaaa2ce8d332b"}
Nov 03 20:44:22.707184 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:22.707155    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"
Nov 03 20:44:22.766026 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:22.766003    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b53b8830-8d79-41aa-a8a3-29bad57a7100-kube-api-access-8qtv6\" (UniqueName: \"kubernetes.io/projected/b53b8830-8d79-41aa-a8a3-29bad57a7100-kube-api-access-8qtv6\") pod \"b53b8830-8d79-41aa-a8a3-29bad57a7100\" (UID: \"b53b8830-8d79-41aa-a8a3-29bad57a7100\") "
Nov 03 20:44:22.766156 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:22.766108    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b53b8830-8d79-41aa-a8a3-29bad57a7100-util\" (UniqueName: \"kubernetes.io/empty-dir/b53b8830-8d79-41aa-a8a3-29bad57a7100-util\") pod \"b53b8830-8d79-41aa-a8a3-29bad57a7100\" (UID: \"b53b8830-8d79-41aa-a8a3-29bad57a7100\") "
Nov 03 20:44:22.766223 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:22.766154    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b53b8830-8d79-41aa-a8a3-29bad57a7100-bundle\" (UniqueName: \"kubernetes.io/empty-dir/b53b8830-8d79-41aa-a8a3-29bad57a7100-bundle\") pod \"b53b8830-8d79-41aa-a8a3-29bad57a7100\" (UID: \"b53b8830-8d79-41aa-a8a3-29bad57a7100\") "
Nov 03 20:44:22.766804 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:22.766778    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b53b8830-8d79-41aa-a8a3-29bad57a7100-bundle" pod "b53b8830-8d79-41aa-a8a3-29bad57a7100" (UID: "b53b8830-8d79-41aa-a8a3-29bad57a7100"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:22.768199 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:22.768171    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b53b8830-8d79-41aa-a8a3-29bad57a7100-kube-api-access-8qtv6" pod "b53b8830-8d79-41aa-a8a3-29bad57a7100" (UID: "b53b8830-8d79-41aa-a8a3-29bad57a7100"). InnerVolumeSpecName "kube-api-access-8qtv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:44:22.771398 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:22.771378    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b53b8830-8d79-41aa-a8a3-29bad57a7100-util" pod "b53b8830-8d79-41aa-a8a3-29bad57a7100" (UID: "b53b8830-8d79-41aa-a8a3-29bad57a7100"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:44:22.867055 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:22.866986    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qtv6\" (UniqueName: \"kubernetes.io/projected/b53b8830-8d79-41aa-a8a3-29bad57a7100-kube-api-access-8qtv6\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:22.867055 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:22.867012    4993 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b53b8830-8d79-41aa-a8a3-29bad57a7100-util\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:22.867055 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:22.867026    4993 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b53b8830-8d79-41aa-a8a3-29bad57a7100-bundle\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:44:23.592648 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:23.592601    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh" event={"ID":"b53b8830-8d79-41aa-a8a3-29bad57a7100","Type":"ContainerDied","Data":"bca885cd4287f5379543d38199e11ac2c817b8d47caa3ad8358f4d25996f840f"}
Nov 03 20:44:23.592773 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:23.592659    4993 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bca885cd4287f5379543d38199e11ac2c817b8d47caa3ad8358f4d25996f840f"
Nov 03 20:44:23.592773 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:23.592659    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ak9xkh"
Nov 03 20:44:31.585542 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:31.585516    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-78789f7d7-xvq57"
Nov 03 20:44:31.668758 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:44:31.668720    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:44:44.907514 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:44.907434    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-zkjdp"]
Nov 03 20:44:44.907901 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:44.907742    4993 cpu_manager.go:414] "containerMap: removing container" podUID="b53b8830-8d79-41aa-a8a3-29bad57a7100" containerName="util"
Nov 03 20:44:44.907901 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:44.907754    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b53b8830-8d79-41aa-a8a3-29bad57a7100" containerName="util"
Nov 03 20:44:44.907901 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:44.907768    4993 cpu_manager.go:414] "containerMap: removing container" podUID="b53b8830-8d79-41aa-a8a3-29bad57a7100" containerName="pull"
Nov 03 20:44:44.907901 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:44.907773    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b53b8830-8d79-41aa-a8a3-29bad57a7100" containerName="pull"
Nov 03 20:44:44.907901 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:44.907793    4993 cpu_manager.go:414] "containerMap: removing container" podUID="b53b8830-8d79-41aa-a8a3-29bad57a7100" containerName="extract"
Nov 03 20:44:44.907901 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:44.907797    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b53b8830-8d79-41aa-a8a3-29bad57a7100" containerName="extract"
Nov 03 20:44:44.907901 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:44.907845    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="b53b8830-8d79-41aa-a8a3-29bad57a7100" containerName="extract"
Nov 03 20:44:44.909724 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:44.909701    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-zkjdp"
Nov 03 20:44:44.932475 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:44.932450    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxwm8\" (UniqueName: \"kubernetes.io/projected/5273aca2-0270-419a-8665-b6354bd7ea9a-kube-api-access-kxwm8\") pod \"authorino-operator-7b7bb4bc77-zkjdp\" (UID: \"5273aca2-0270-419a-8665-b6354bd7ea9a\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-zkjdp"
Nov 03 20:44:45.015419 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:45.015393    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 03 20:44:45.015587 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:45.015466    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 03 20:44:45.015587 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:45.015486    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-sdprx\""
Nov 03 20:44:45.016694 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:45.016676    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-zkjdp"]
Nov 03 20:44:45.033266 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:45.033247    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxwm8\" (UniqueName: \"kubernetes.io/projected/5273aca2-0270-419a-8665-b6354bd7ea9a-kube-api-access-kxwm8\") pod \"authorino-operator-7b7bb4bc77-zkjdp\" (UID: \"5273aca2-0270-419a-8665-b6354bd7ea9a\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-zkjdp"
Nov 03 20:44:45.101899 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:45.101872    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxwm8\" (UniqueName: \"kubernetes.io/projected/5273aca2-0270-419a-8665-b6354bd7ea9a-kube-api-access-kxwm8\") pod \"authorino-operator-7b7bb4bc77-zkjdp\" (UID: \"5273aca2-0270-419a-8665-b6354bd7ea9a\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-zkjdp"
Nov 03 20:44:45.221503 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:45.221255    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-zkjdp"
Nov 03 20:44:45.630613 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:45.630541    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-zkjdp"]
Nov 03 20:44:45.633766 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:44:45.633739    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5273aca2_0270_419a_8665_b6354bd7ea9a.slice/crio-d57ad8f91a1cd4b7a12353a5006597031007ae7c60724174c9f4d06172a37214 WatchSource:0}: Error finding container d57ad8f91a1cd4b7a12353a5006597031007ae7c60724174c9f4d06172a37214: Status 404 returned error can't find the container with id d57ad8f91a1cd4b7a12353a5006597031007ae7c60724174c9f4d06172a37214
Nov 03 20:44:45.680110 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:45.680082    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-zkjdp" event={"ID":"5273aca2-0270-419a-8665-b6354bd7ea9a","Type":"ContainerStarted","Data":"d57ad8f91a1cd4b7a12353a5006597031007ae7c60724174c9f4d06172a37214"}
Nov 03 20:44:49.698712 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:49.698679    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-zkjdp" event={"ID":"5273aca2-0270-419a-8665-b6354bd7ea9a","Type":"ContainerStarted","Data":"a59b9d98ba6087c244185eda2a900368b99e09b3731e15f5f2157941542ae1a4"}
Nov 03 20:44:49.699093 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:49.698823    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-zkjdp"
Nov 03 20:44:49.823429 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:49.823365    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-zkjdp" podStartSLOduration=2.639236084 podStartE2EDuration="5.823350302s" totalImagesPullingTime="3.184114218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:44:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:44:49.822936626 +0000 UTC m=+292.367642997" watchObservedRunningTime="2026-11-03 20:44:49.823350302 +0000 UTC m=+292.368056673"
Nov 03 20:44:56.773292 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:56.773265    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"]
Nov 03 20:44:56.779400 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:56.779379    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"
Nov 03 20:44:56.829870 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:56.829836    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzmhp\" (UniqueName: \"kubernetes.io/projected/2ff49561-6d05-43e2-9a27-3fa48f5be683-kube-api-access-kzmhp\") pod \"kuadrant-operator-controller-manager-678d87766f-f9dzg\" (UID: \"2ff49561-6d05-43e2-9a27-3fa48f5be683\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"
Nov 03 20:44:56.830015 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:56.830001    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ff49561-6d05-43e2-9a27-3fa48f5be683-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-f9dzg\" (UID: \"2ff49561-6d05-43e2-9a27-3fa48f5be683\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"
Nov 03 20:44:56.830738 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:56.830717    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"]
Nov 03 20:44:56.879258 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:56.879235    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6p7qs\""
Nov 03 20:44:56.931316 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:56.931290    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ff49561-6d05-43e2-9a27-3fa48f5be683-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-f9dzg\" (UID: \"2ff49561-6d05-43e2-9a27-3fa48f5be683\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"
Nov 03 20:44:56.931442 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:56.931342    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzmhp\" (UniqueName: \"kubernetes.io/projected/2ff49561-6d05-43e2-9a27-3fa48f5be683-kube-api-access-kzmhp\") pod \"kuadrant-operator-controller-manager-678d87766f-f9dzg\" (UID: \"2ff49561-6d05-43e2-9a27-3fa48f5be683\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"
Nov 03 20:44:56.931685 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:56.931668    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ff49561-6d05-43e2-9a27-3fa48f5be683-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-f9dzg\" (UID: \"2ff49561-6d05-43e2-9a27-3fa48f5be683\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"
Nov 03 20:44:56.994836 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:56.994801    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzmhp\" (UniqueName: \"kubernetes.io/projected/2ff49561-6d05-43e2-9a27-3fa48f5be683-kube-api-access-kzmhp\") pod \"kuadrant-operator-controller-manager-678d87766f-f9dzg\" (UID: \"2ff49561-6d05-43e2-9a27-3fa48f5be683\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"
Nov 03 20:44:57.090131 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:57.090062    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"
Nov 03 20:44:57.562872 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:57.562845    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"]
Nov 03 20:44:57.565316 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:44:57.565289    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ff49561_6d05_43e2_9a27_3fa48f5be683.slice/crio-f0734f3ed521553e9e3c44b9b086fa6a91ec4f27edc9ca1e6c1e2718a643e1f0 WatchSource:0}: Error finding container f0734f3ed521553e9e3c44b9b086fa6a91ec4f27edc9ca1e6c1e2718a643e1f0: Status 404 returned error can't find the container with id f0734f3ed521553e9e3c44b9b086fa6a91ec4f27edc9ca1e6c1e2718a643e1f0
Nov 03 20:44:57.730302 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:44:57.730251    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg" event={"ID":"2ff49561-6d05-43e2-9a27-3fa48f5be683","Type":"ContainerStarted","Data":"f0734f3ed521553e9e3c44b9b086fa6a91ec4f27edc9ca1e6c1e2718a643e1f0"}
Nov 03 20:45:00.705716 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:00.705680    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-zkjdp"
Nov 03 20:45:01.674380 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:45:01.674340    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:45:02.755136 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:02.755100    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg" event={"ID":"2ff49561-6d05-43e2-9a27-3fa48f5be683","Type":"ContainerStarted","Data":"2c138666cee6fcfb86cc20edc43ef2808aaf35809115341c8e4b3eebc65d30fe"}
Nov 03 20:45:02.755534 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:02.755186    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"
Nov 03 20:45:02.973785 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:02.973729    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg" podStartSLOduration=2.680430743 podStartE2EDuration="6.973714605s" totalImagesPullingTime="4.293283862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:44:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:45:02.972537741 +0000 UTC m=+305.517244111" watchObservedRunningTime="2026-11-03 20:45:02.973714605 +0000 UTC m=+305.518420975"
Nov 03 20:45:08.364812 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.364776    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"]
Nov 03 20:45:08.368132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.368111    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:08.420688 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.420657    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 03 20:45:08.469880 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.469852    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 03 20:45:08.470026 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.469955    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Nov 03 20:45:08.470026 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.469959    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Nov 03 20:45:08.470115 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.470066    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Nov 03 20:45:08.470115 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.470066    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-4bvbl\""
Nov 03 20:45:08.472322 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.472295    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"]
Nov 03 20:45:08.532169 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.532144    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a792812a-09e8-45eb-b1e1-35a257044623-manager-config\") pod \"jobset-controller-manager-5dc7d655c6-snzc4\" (UID: \"a792812a-09e8-45eb-b1e1-35a257044623\") " pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:08.532294 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.532193    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a792812a-09e8-45eb-b1e1-35a257044623-metrics-certs\") pod \"jobset-controller-manager-5dc7d655c6-snzc4\" (UID: \"a792812a-09e8-45eb-b1e1-35a257044623\") " pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:08.532294 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.532268    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a792812a-09e8-45eb-b1e1-35a257044623-cert\") pod \"jobset-controller-manager-5dc7d655c6-snzc4\" (UID: \"a792812a-09e8-45eb-b1e1-35a257044623\") " pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:08.532372 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.532301    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk4t8\" (UniqueName: \"kubernetes.io/projected/a792812a-09e8-45eb-b1e1-35a257044623-kube-api-access-lk4t8\") pod \"jobset-controller-manager-5dc7d655c6-snzc4\" (UID: \"a792812a-09e8-45eb-b1e1-35a257044623\") " pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:08.633648 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.633549    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a792812a-09e8-45eb-b1e1-35a257044623-manager-config\") pod \"jobset-controller-manager-5dc7d655c6-snzc4\" (UID: \"a792812a-09e8-45eb-b1e1-35a257044623\") " pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:08.633777 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.633622    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a792812a-09e8-45eb-b1e1-35a257044623-metrics-certs\") pod \"jobset-controller-manager-5dc7d655c6-snzc4\" (UID: \"a792812a-09e8-45eb-b1e1-35a257044623\") " pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:08.633777 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.633689    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a792812a-09e8-45eb-b1e1-35a257044623-cert\") pod \"jobset-controller-manager-5dc7d655c6-snzc4\" (UID: \"a792812a-09e8-45eb-b1e1-35a257044623\") " pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:08.633777 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.633717    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk4t8\" (UniqueName: \"kubernetes.io/projected/a792812a-09e8-45eb-b1e1-35a257044623-kube-api-access-lk4t8\") pod \"jobset-controller-manager-5dc7d655c6-snzc4\" (UID: \"a792812a-09e8-45eb-b1e1-35a257044623\") " pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:08.634210 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.634186    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a792812a-09e8-45eb-b1e1-35a257044623-manager-config\") pod \"jobset-controller-manager-5dc7d655c6-snzc4\" (UID: \"a792812a-09e8-45eb-b1e1-35a257044623\") " pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:08.636199 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.636177    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a792812a-09e8-45eb-b1e1-35a257044623-metrics-certs\") pod \"jobset-controller-manager-5dc7d655c6-snzc4\" (UID: \"a792812a-09e8-45eb-b1e1-35a257044623\") " pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:08.636294 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.636276    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a792812a-09e8-45eb-b1e1-35a257044623-cert\") pod \"jobset-controller-manager-5dc7d655c6-snzc4\" (UID: \"a792812a-09e8-45eb-b1e1-35a257044623\") " pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:08.694430 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.694401    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk4t8\" (UniqueName: \"kubernetes.io/projected/a792812a-09e8-45eb-b1e1-35a257044623-kube-api-access-lk4t8\") pod \"jobset-controller-manager-5dc7d655c6-snzc4\" (UID: \"a792812a-09e8-45eb-b1e1-35a257044623\") " pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:08.979567 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:08.979524    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:09.348511 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:09.348413    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"]
Nov 03 20:45:09.351221 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:45:09.351194    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda792812a_09e8_45eb_b1e1_35a257044623.slice/crio-b1a709b7221694a19cc8e60528f7c7e16c84081d12f7c80efd0354c799a8a4bd WatchSource:0}: Error finding container b1a709b7221694a19cc8e60528f7c7e16c84081d12f7c80efd0354c799a8a4bd: Status 404 returned error can't find the container with id b1a709b7221694a19cc8e60528f7c7e16c84081d12f7c80efd0354c799a8a4bd
Nov 03 20:45:09.353150 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:09.353135    4993 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 20:45:09.784300 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:09.784269    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4" event={"ID":"a792812a-09e8-45eb-b1e1-35a257044623","Type":"ContainerStarted","Data":"b1a709b7221694a19cc8e60528f7c7e16c84081d12f7c80efd0354c799a8a4bd"}
Nov 03 20:45:13.761797 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:13.761762    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"
Nov 03 20:45:13.804318 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:13.804284    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4" event={"ID":"a792812a-09e8-45eb-b1e1-35a257044623","Type":"ContainerStarted","Data":"db0a0ea93131814e6cc03fbb61770ae74e2083e426c55a02f8ba523fe4a5a754"}
Nov 03 20:45:13.804464 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:13.804337    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:14.045582 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:14.045480    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4" podStartSLOduration=2.072209914 podStartE2EDuration="6.045464585s" totalImagesPullingTime="3.973254671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:45:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:45:14.044846092 +0000 UTC m=+316.589552462" watchObservedRunningTime="2026-11-03 20:45:14.045464585 +0000 UTC m=+316.590170955"
Nov 03 20:45:24.813322 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:24.813285    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-5dc7d655c6-snzc4"
Nov 03 20:45:31.668818 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:45:31.668783    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:45:36.255032 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:36.255002    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9"]
Nov 03 20:45:36.257191 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:36.257171    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9"
Nov 03 20:45:36.315018 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:36.314994    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 03 20:45:36.340383 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:36.340352    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjxwj\" (UniqueName: \"kubernetes.io/projected/c3ad1a5d-d272-43d5-9e6c-0c18e37f6203-kube-api-access-tjxwj\") pod \"postgres-7cdbf6dc44-85hv9\" (UID: \"c3ad1a5d-d272-43d5-9e6c-0c18e37f6203\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9"
Nov 03 20:45:36.340514 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:36.340428    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c3ad1a5d-d272-43d5-9e6c-0c18e37f6203-data\") pod \"postgres-7cdbf6dc44-85hv9\" (UID: \"c3ad1a5d-d272-43d5-9e6c-0c18e37f6203\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9"
Nov 03 20:45:36.360322 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:36.360303    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Nov 03 20:45:36.360322 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:36.360303    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 03 20:45:36.360456 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:36.360309    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-2vsb7\""
Nov 03 20:45:36.361228 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:36.361209    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9"]
Nov 03 20:45:36.441475 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:36.441450    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjxwj\" (UniqueName: \"kubernetes.io/projected/c3ad1a5d-d272-43d5-9e6c-0c18e37f6203-kube-api-access-tjxwj\") pod \"postgres-7cdbf6dc44-85hv9\" (UID: \"c3ad1a5d-d272-43d5-9e6c-0c18e37f6203\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9"
Nov 03 20:45:36.441590 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:36.441503    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c3ad1a5d-d272-43d5-9e6c-0c18e37f6203-data\") pod \"postgres-7cdbf6dc44-85hv9\" (UID: \"c3ad1a5d-d272-43d5-9e6c-0c18e37f6203\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9"
Nov 03 20:45:36.441818 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:36.441803    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c3ad1a5d-d272-43d5-9e6c-0c18e37f6203-data\") pod \"postgres-7cdbf6dc44-85hv9\" (UID: \"c3ad1a5d-d272-43d5-9e6c-0c18e37f6203\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9"
Nov 03 20:45:36.512809 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:36.512740    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjxwj\" (UniqueName: \"kubernetes.io/projected/c3ad1a5d-d272-43d5-9e6c-0c18e37f6203-kube-api-access-tjxwj\") pod \"postgres-7cdbf6dc44-85hv9\" (UID: \"c3ad1a5d-d272-43d5-9e6c-0c18e37f6203\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9"
Nov 03 20:45:36.569348 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:36.569322    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9"
Nov 03 20:45:36.952602 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:36.952524    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9"]
Nov 03 20:45:36.955561 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:45:36.955532    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3ad1a5d_d272_43d5_9e6c_0c18e37f6203.slice/crio-887cf71398de93162ef7e1978ee366da9f6638308f89624447c721c75ec48e12 WatchSource:0}: Error finding container 887cf71398de93162ef7e1978ee366da9f6638308f89624447c721c75ec48e12: Status 404 returned error can't find the container with id 887cf71398de93162ef7e1978ee366da9f6638308f89624447c721c75ec48e12
Nov 03 20:45:37.905305 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:37.905269    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9" event={"ID":"c3ad1a5d-d272-43d5-9e6c-0c18e37f6203","Type":"ContainerStarted","Data":"887cf71398de93162ef7e1978ee366da9f6638308f89624447c721c75ec48e12"}
Nov 03 20:45:44.937129 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:44.937090    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9" event={"ID":"c3ad1a5d-d272-43d5-9e6c-0c18e37f6203","Type":"ContainerStarted","Data":"105ca367f6db9a3dac3e1ad2af95253adb0ddfd1d2237ade4282288d38b7dd0b"}
Nov 03 20:45:44.937511 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:44.937229    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9"
Nov 03 20:45:45.053115 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:45.053054    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9" podStartSLOduration=1.6378967210000002 podStartE2EDuration="9.053039011s" totalImagesPullingTime="7.41514229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:45:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:45:45.053024619 +0000 UTC m=+347.597730989" watchObservedRunningTime="2026-11-03 20:45:45.053039011 +0000 UTC m=+347.597745385"
Nov 03 20:45:50.971194 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:45:50.971168    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-85hv9"
Nov 03 20:46:01.669830 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:46:01.669794    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:46:28.612747 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.612712    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-4fm4w"]
Nov 03 20:46:28.628557 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.628536    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-4fm4w"
Nov 03 20:46:28.674259 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.674234    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-4fm4w"]
Nov 03 20:46:28.722114 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.722090    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 03 20:46:28.722270 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.722255    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 03 20:46:28.730026 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.730004    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-rbp8j\""
Nov 03 20:46:28.730276 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.730260    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Nov 03 20:46:28.731436 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.731420    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-5vf4s"]
Nov 03 20:46:28.733539 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.733525    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-5vf4s"
Nov 03 20:46:28.766804 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.766780    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9hmm\" (UniqueName: \"kubernetes.io/projected/5b525af5-eb4e-4ca6-bca2-71eb924f1a30-kube-api-access-z9hmm\") pod \"model-serving-api-75bd89fdfc-4fm4w\" (UID: \"5b525af5-eb4e-4ca6-bca2-71eb924f1a30\") " pod="opendatahub/model-serving-api-75bd89fdfc-4fm4w"
Nov 03 20:46:28.766902 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.766820    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b525af5-eb4e-4ca6-bca2-71eb924f1a30-tls-certs\") pod \"model-serving-api-75bd89fdfc-4fm4w\" (UID: \"5b525af5-eb4e-4ca6-bca2-71eb924f1a30\") " pod="opendatahub/model-serving-api-75bd89fdfc-4fm4w"
Nov 03 20:46:28.788162 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.788134    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-wglpp\""
Nov 03 20:46:28.788283 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.788170    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Nov 03 20:46:28.788283 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.788211    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Nov 03 20:46:28.789614 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.789595    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-5vf4s"]
Nov 03 20:46:28.867846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.867788    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f18b897-8328-469a-a7ec-89132fde8f95-cert\") pod \"odh-model-controller-769875db56-5vf4s\" (UID: \"4f18b897-8328-469a-a7ec-89132fde8f95\") " pod="opendatahub/odh-model-controller-769875db56-5vf4s"
Nov 03 20:46:28.867846 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.867826    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9hmm\" (UniqueName: \"kubernetes.io/projected/5b525af5-eb4e-4ca6-bca2-71eb924f1a30-kube-api-access-z9hmm\") pod \"model-serving-api-75bd89fdfc-4fm4w\" (UID: \"5b525af5-eb4e-4ca6-bca2-71eb924f1a30\") " pod="opendatahub/model-serving-api-75bd89fdfc-4fm4w"
Nov 03 20:46:28.868026 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.867848    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2lgs\" (UniqueName: \"kubernetes.io/projected/4f18b897-8328-469a-a7ec-89132fde8f95-kube-api-access-q2lgs\") pod \"odh-model-controller-769875db56-5vf4s\" (UID: \"4f18b897-8328-469a-a7ec-89132fde8f95\") " pod="opendatahub/odh-model-controller-769875db56-5vf4s"
Nov 03 20:46:28.868026 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.867933    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b525af5-eb4e-4ca6-bca2-71eb924f1a30-tls-certs\") pod \"model-serving-api-75bd89fdfc-4fm4w\" (UID: \"5b525af5-eb4e-4ca6-bca2-71eb924f1a30\") " pod="opendatahub/model-serving-api-75bd89fdfc-4fm4w"
Nov 03 20:46:28.868026 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.867974    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f18b897-8328-469a-a7ec-89132fde8f95-metrics-certs\") pod \"odh-model-controller-769875db56-5vf4s\" (UID: \"4f18b897-8328-469a-a7ec-89132fde8f95\") " pod="opendatahub/odh-model-controller-769875db56-5vf4s"
Nov 03 20:46:28.870557 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.870535    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b525af5-eb4e-4ca6-bca2-71eb924f1a30-tls-certs\") pod \"model-serving-api-75bd89fdfc-4fm4w\" (UID: \"5b525af5-eb4e-4ca6-bca2-71eb924f1a30\") " pod="opendatahub/model-serving-api-75bd89fdfc-4fm4w"
Nov 03 20:46:28.928744 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.928717    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9hmm\" (UniqueName: \"kubernetes.io/projected/5b525af5-eb4e-4ca6-bca2-71eb924f1a30-kube-api-access-z9hmm\") pod \"model-serving-api-75bd89fdfc-4fm4w\" (UID: \"5b525af5-eb4e-4ca6-bca2-71eb924f1a30\") " pod="opendatahub/model-serving-api-75bd89fdfc-4fm4w"
Nov 03 20:46:28.940510 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.940487    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-4fm4w"
Nov 03 20:46:28.968761 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.968740    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f18b897-8328-469a-a7ec-89132fde8f95-cert\") pod \"odh-model-controller-769875db56-5vf4s\" (UID: \"4f18b897-8328-469a-a7ec-89132fde8f95\") " pod="opendatahub/odh-model-controller-769875db56-5vf4s"
Nov 03 20:46:28.968876 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.968777    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2lgs\" (UniqueName: \"kubernetes.io/projected/4f18b897-8328-469a-a7ec-89132fde8f95-kube-api-access-q2lgs\") pod \"odh-model-controller-769875db56-5vf4s\" (UID: \"4f18b897-8328-469a-a7ec-89132fde8f95\") " pod="opendatahub/odh-model-controller-769875db56-5vf4s"
Nov 03 20:46:28.968876 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.968816    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f18b897-8328-469a-a7ec-89132fde8f95-metrics-certs\") pod \"odh-model-controller-769875db56-5vf4s\" (UID: \"4f18b897-8328-469a-a7ec-89132fde8f95\") " pod="opendatahub/odh-model-controller-769875db56-5vf4s"
Nov 03 20:46:28.971125 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.971102    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f18b897-8328-469a-a7ec-89132fde8f95-cert\") pod \"odh-model-controller-769875db56-5vf4s\" (UID: \"4f18b897-8328-469a-a7ec-89132fde8f95\") " pod="opendatahub/odh-model-controller-769875db56-5vf4s"
Nov 03 20:46:28.971280 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:28.971261    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f18b897-8328-469a-a7ec-89132fde8f95-metrics-certs\") pod \"odh-model-controller-769875db56-5vf4s\" (UID: \"4f18b897-8328-469a-a7ec-89132fde8f95\") " pod="opendatahub/odh-model-controller-769875db56-5vf4s"
Nov 03 20:46:29.029327 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:29.029304    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2lgs\" (UniqueName: \"kubernetes.io/projected/4f18b897-8328-469a-a7ec-89132fde8f95-kube-api-access-q2lgs\") pod \"odh-model-controller-769875db56-5vf4s\" (UID: \"4f18b897-8328-469a-a7ec-89132fde8f95\") " pod="opendatahub/odh-model-controller-769875db56-5vf4s"
Nov 03 20:46:29.046196 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:29.046167    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-5vf4s"
Nov 03 20:46:29.307331 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:29.307306    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-4fm4w"]
Nov 03 20:46:29.310330 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:46:29.310302    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b525af5_eb4e_4ca6_bca2_71eb924f1a30.slice/crio-59590e5462fca6da27084f404518e64d776723136a3ee94eeff386f6972f8cbc WatchSource:0}: Error finding container 59590e5462fca6da27084f404518e64d776723136a3ee94eeff386f6972f8cbc: Status 404 returned error can't find the container with id 59590e5462fca6da27084f404518e64d776723136a3ee94eeff386f6972f8cbc
Nov 03 20:46:29.415035 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:29.414885    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-5vf4s"]
Nov 03 20:46:29.417434 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:46:29.417403    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f18b897_8328_469a_a7ec_89132fde8f95.slice/crio-cb85774ef30c5051ec4c0181db94d1dad58d815ba2ac532643d47a8ba2300115 WatchSource:0}: Error finding container cb85774ef30c5051ec4c0181db94d1dad58d815ba2ac532643d47a8ba2300115: Status 404 returned error can't find the container with id cb85774ef30c5051ec4c0181db94d1dad58d815ba2ac532643d47a8ba2300115
Nov 03 20:46:30.124207 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:30.124171    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-4fm4w" event={"ID":"5b525af5-eb4e-4ca6-bca2-71eb924f1a30","Type":"ContainerStarted","Data":"59590e5462fca6da27084f404518e64d776723136a3ee94eeff386f6972f8cbc"}
Nov 03 20:46:30.125504 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:30.125479    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-5vf4s" event={"ID":"4f18b897-8328-469a-a7ec-89132fde8f95","Type":"ContainerStarted","Data":"cb85774ef30c5051ec4c0181db94d1dad58d815ba2ac532643d47a8ba2300115"}
Nov 03 20:46:31.680475 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:46:31.680433    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:46:32.135290 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:32.135259    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-4fm4w" event={"ID":"5b525af5-eb4e-4ca6-bca2-71eb924f1a30","Type":"ContainerStarted","Data":"96b45eb7cee83d96a262c0f1fa832a1388cfbf485cefe3518870c2fa5b3f2638"}
Nov 03 20:46:32.135530 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:32.135508    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-4fm4w"
Nov 03 20:46:32.252866 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:32.251687    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-4fm4w" podStartSLOduration=1.745505896 podStartE2EDuration="4.251669353s" totalImagesPullingTime="2.506163457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:46:32.251047265 +0000 UTC m=+394.795753637" watchObservedRunningTime="2026-11-03 20:46:32.251669353 +0000 UTC m=+394.796375724"
Nov 03 20:46:33.141087 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:33.141056    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-5vf4s" event={"ID":"4f18b897-8328-469a-a7ec-89132fde8f95","Type":"ContainerStarted","Data":"adf57a1400dfd50fc99d9f052088be1efd10d5700951531e9ef7c8c53666dc6d"}
Nov 03 20:46:33.141457 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:33.141241    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-5vf4s"
Nov 03 20:46:33.267131 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:33.267048    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-5vf4s" podStartSLOduration=1.695215883 podStartE2EDuration="5.267035692s" totalImagesPullingTime="3.571819809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:46:33.266730873 +0000 UTC m=+395.811437243" watchObservedRunningTime="2026-11-03 20:46:33.267035692 +0000 UTC m=+395.811742062"
Nov 03 20:46:35.725651 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.725599    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"]
Nov 03 20:46:35.728890 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.728860    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:35.729255 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.729219    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3966dfa8-e2af-4ff5-a076-15835ca00fca-maas-api-tls\") pod \"maas-api-6d669c7fcd-krcb7\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:35.729328 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.729271    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm7bd\" (UniqueName: \"kubernetes.io/projected/3966dfa8-e2af-4ff5-a076-15835ca00fca-kube-api-access-hm7bd\") pod \"maas-api-6d669c7fcd-krcb7\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:35.729406 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.729392    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3966dfa8-e2af-4ff5-a076-15835ca00fca-metrics-certs\") pod \"maas-api-6d669c7fcd-krcb7\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:35.729449 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.729425    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3966dfa8-e2af-4ff5-a076-15835ca00fca-openshift-service-ca\") pod \"maas-api-6d669c7fcd-krcb7\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:35.830420 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.830387    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3966dfa8-e2af-4ff5-a076-15835ca00fca-maas-api-tls\") pod \"maas-api-6d669c7fcd-krcb7\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:35.830565 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.830424    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm7bd\" (UniqueName: \"kubernetes.io/projected/3966dfa8-e2af-4ff5-a076-15835ca00fca-kube-api-access-hm7bd\") pod \"maas-api-6d669c7fcd-krcb7\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:35.830565 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.830495    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3966dfa8-e2af-4ff5-a076-15835ca00fca-metrics-certs\") pod \"maas-api-6d669c7fcd-krcb7\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:35.830565 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.830534    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3966dfa8-e2af-4ff5-a076-15835ca00fca-openshift-service-ca\") pod \"maas-api-6d669c7fcd-krcb7\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:35.831166 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.831145    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3966dfa8-e2af-4ff5-a076-15835ca00fca-openshift-service-ca\") pod \"maas-api-6d669c7fcd-krcb7\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:35.832458 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.832436    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 03 20:46:35.832662 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.832648    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lnz6c\""
Nov 03 20:46:35.832705 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.832654    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 03 20:46:35.834761 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.834743    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"]
Nov 03 20:46:35.842975 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.842953    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3966dfa8-e2af-4ff5-a076-15835ca00fca-maas-api-tls\") pod \"maas-api-6d669c7fcd-krcb7\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:35.842975 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.842969    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3966dfa8-e2af-4ff5-a076-15835ca00fca-metrics-certs\") pod \"maas-api-6d669c7fcd-krcb7\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:35.891097 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:35.891077    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm7bd\" (UniqueName: \"kubernetes.io/projected/3966dfa8-e2af-4ff5-a076-15835ca00fca-kube-api-access-hm7bd\") pod \"maas-api-6d669c7fcd-krcb7\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:36.041314 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.041244    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:36.329296 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.329223    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"]
Nov 03 20:46:36.329452 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.329431    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4" podUID="7fd09aa6-d1f6-4f41-8eab-fac00f97938e" containerName="discovery" containerID="cri-o://a2e2644ba8e1e51bc8061f3ab46a483c81d5706668cce9c81da60bbd0061814d" gracePeriod=30
Nov 03 20:46:36.365050 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.365022    4993 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-2rjp4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.17:8080/ready\": dial tcp 10.133.0.17:8080: connect: connection refused" start-of-body=
Nov 03 20:46:36.365171 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.365065    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4" podUID="7fd09aa6-d1f6-4f41-8eab-fac00f97938e" containerName="discovery" probeResult="failure" output="Get \"http://10.133.0.17:8080/ready\": dial tcp 10.133.0.17:8080: connect: connection refused"
Nov 03 20:46:36.442885 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.442814    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"]
Nov 03 20:46:36.470694 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:46:36.470660    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3966dfa8_e2af_4ff5_a076_15835ca00fca.slice/crio-9b610b55e70ea3425cc17024e37dae83de92f5bbb14a5bbbb7a85d0be8a8a24b WatchSource:0}: Error finding container 9b610b55e70ea3425cc17024e37dae83de92f5bbb14a5bbbb7a85d0be8a8a24b: Status 404 returned error can't find the container with id 9b610b55e70ea3425cc17024e37dae83de92f5bbb14a5bbbb7a85d0be8a8a24b
Nov 03 20:46:36.572233 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.572210    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:46:36.737146 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.737116    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-local-certs\") pod \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") "
Nov 03 20:46:36.737573 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.737156    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-cacerts\" (UniqueName: \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-cacerts\") pod \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") "
Nov 03 20:46:36.737573 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.737207    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-kubeconfig\") pod \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") "
Nov 03 20:46:36.737573 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.737225    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-kube-api-access-6xdpg\" (UniqueName: \"kubernetes.io/projected/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-kube-api-access-6xdpg\") pod \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") "
Nov 03 20:46:36.737573 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.737244    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-token\" (UniqueName: \"kubernetes.io/projected/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-token\") pod \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") "
Nov 03 20:46:36.737573 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.737264    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-csr-dns-cert\") pod \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") "
Nov 03 20:46:36.737573 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.737343    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-csr-ca-configmap\") pod \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\" (UID: \"7fd09aa6-d1f6-4f41-8eab-fac00f97938e\") "
Nov 03 20:46:36.738123 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.738075    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-csr-ca-configmap" pod "7fd09aa6-d1f6-4f41-8eab-fac00f97938e" (UID: "7fd09aa6-d1f6-4f41-8eab-fac00f97938e"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:46:36.739526 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.739498    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-local-certs" pod "7fd09aa6-d1f6-4f41-8eab-fac00f97938e" (UID: "7fd09aa6-d1f6-4f41-8eab-fac00f97938e"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 20:46:36.739703 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.739654    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-kube-api-access-6xdpg" pod "7fd09aa6-d1f6-4f41-8eab-fac00f97938e" (UID: "7fd09aa6-d1f6-4f41-8eab-fac00f97938e"). InnerVolumeSpecName "kube-api-access-6xdpg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:46:36.739956 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.739933    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-kubeconfig" pod "7fd09aa6-d1f6-4f41-8eab-fac00f97938e" (UID: "7fd09aa6-d1f6-4f41-8eab-fac00f97938e"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:46:36.740041 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.740000    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-csr-dns-cert" pod "7fd09aa6-d1f6-4f41-8eab-fac00f97938e" (UID: "7fd09aa6-d1f6-4f41-8eab-fac00f97938e"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:46:36.740131 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.740115    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-cacerts" pod "7fd09aa6-d1f6-4f41-8eab-fac00f97938e" (UID: "7fd09aa6-d1f6-4f41-8eab-fac00f97938e"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:46:36.740185 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.740126    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-token" pod "7fd09aa6-d1f6-4f41-8eab-fac00f97938e" (UID: "7fd09aa6-d1f6-4f41-8eab-fac00f97938e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:46:36.837953 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.837920    4993 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-kubeconfig\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:46:36.837953 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.837950    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xdpg\" (UniqueName: \"kubernetes.io/projected/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-kube-api-access-6xdpg\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:46:36.838131 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.837965    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:46:36.838131 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.837977    4993 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-csr-dns-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:46:36.838131 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.837989    4993 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-istio-csr-ca-configmap\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:46:36.838131 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.838001    4993 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-local-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:46:36.838131 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:36.838013    4993 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7fd09aa6-d1f6-4f41-8eab-fac00f97938e-cacerts\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:46:37.161170 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:37.161084    4993 generic.go:356] "Generic (PLEG): container finished" podID="7fd09aa6-d1f6-4f41-8eab-fac00f97938e" containerID="a2e2644ba8e1e51bc8061f3ab46a483c81d5706668cce9c81da60bbd0061814d" exitCode=0
Nov 03 20:46:37.161170 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:37.161133    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"
Nov 03 20:46:37.161412 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:37.161168    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4" event={"ID":"7fd09aa6-d1f6-4f41-8eab-fac00f97938e","Type":"ContainerDied","Data":"a2e2644ba8e1e51bc8061f3ab46a483c81d5706668cce9c81da60bbd0061814d"}
Nov 03 20:46:37.161412 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:37.161219    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4" event={"ID":"7fd09aa6-d1f6-4f41-8eab-fac00f97938e","Type":"ContainerDied","Data":"013d7461f35404bd2a24a46d5823073923c303451f1f24b07e35c3a0b58e9539"}
Nov 03 20:46:37.161412 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:37.161241    4993 scope.go:122] "RemoveContainer" containerID="a2e2644ba8e1e51bc8061f3ab46a483c81d5706668cce9c81da60bbd0061814d"
Nov 03 20:46:37.165826 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:37.165696    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7" event={"ID":"3966dfa8-e2af-4ff5-a076-15835ca00fca","Type":"ContainerStarted","Data":"9b610b55e70ea3425cc17024e37dae83de92f5bbb14a5bbbb7a85d0be8a8a24b"}
Nov 03 20:46:37.172858 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:37.172783    4993 scope.go:122] "RemoveContainer" containerID="a2e2644ba8e1e51bc8061f3ab46a483c81d5706668cce9c81da60bbd0061814d"
Nov 03 20:46:37.173072 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:46:37.173050    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2e2644ba8e1e51bc8061f3ab46a483c81d5706668cce9c81da60bbd0061814d\": container with ID starting with a2e2644ba8e1e51bc8061f3ab46a483c81d5706668cce9c81da60bbd0061814d not found: ID does not exist" containerID="a2e2644ba8e1e51bc8061f3ab46a483c81d5706668cce9c81da60bbd0061814d"
Nov 03 20:46:37.173137 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:37.173079    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2e2644ba8e1e51bc8061f3ab46a483c81d5706668cce9c81da60bbd0061814d"} err="failed to get container status \"a2e2644ba8e1e51bc8061f3ab46a483c81d5706668cce9c81da60bbd0061814d\": rpc error: code = NotFound desc = could not find container \"a2e2644ba8e1e51bc8061f3ab46a483c81d5706668cce9c81da60bbd0061814d\": container with ID starting with a2e2644ba8e1e51bc8061f3ab46a483c81d5706668cce9c81da60bbd0061814d not found: ID does not exist"
Nov 03 20:46:37.325221 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:37.325189    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"]
Nov 03 20:46:37.329208 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:37.329188    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-2rjp4"]
Nov 03 20:46:37.577146 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:37.577115    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7fd09aa6-d1f6-4f41-8eab-fac00f97938e" path="/var/lib/kubelet/pods/7fd09aa6-d1f6-4f41-8eab-fac00f97938e/volumes"
Nov 03 20:46:40.185965 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:40.185927    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7" event={"ID":"3966dfa8-e2af-4ff5-a076-15835ca00fca","Type":"ContainerStarted","Data":"9e42ce95d0096a12d43e444c3e2f6fc42faa87702769a4efcb65344e79a91b98"}
Nov 03 20:46:40.186419 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:40.186051    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:40.305457 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:40.305404    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7" podStartSLOduration=2.332655673 podStartE2EDuration="5.305391114s" totalImagesPullingTime="2.972735441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:46:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:46:40.304821075 +0000 UTC m=+402.849527446" watchObservedRunningTime="2026-11-03 20:46:40.305391114 +0000 UTC m=+402.850097484"
Nov 03 20:46:43.144965 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:43.144938    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-4fm4w"
Nov 03 20:46:44.162975 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:44.162944    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-5vf4s"
Nov 03 20:46:46.196760 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:46.196734    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:47.203707 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.203676    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"]
Nov 03 20:46:47.204133 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.204116    4993 cpu_manager.go:414] "containerMap: removing container" podUID="7fd09aa6-d1f6-4f41-8eab-fac00f97938e" containerName="discovery"
Nov 03 20:46:47.204176 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.204136    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7fd09aa6-d1f6-4f41-8eab-fac00f97938e" containerName="discovery"
Nov 03 20:46:47.204214 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.204202    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="7fd09aa6-d1f6-4f41-8eab-fac00f97938e" containerName="discovery"
Nov 03 20:46:47.207251 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.207234    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:47.310529 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.310485    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"]
Nov 03 20:46:47.327579 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.327556    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-openshift-service-ca\") pod \"maas-api-8669f899bf-59f8m\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:47.327737 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.327587    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-maas-api-tls\") pod \"maas-api-8669f899bf-59f8m\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:47.327737 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.327609    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xvdt\" (UniqueName: \"kubernetes.io/projected/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-kube-api-access-8xvdt\") pod \"maas-api-8669f899bf-59f8m\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:47.327737 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.327649    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-metrics-certs\") pod \"maas-api-8669f899bf-59f8m\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:47.428485 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.428455    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-openshift-service-ca\") pod \"maas-api-8669f899bf-59f8m\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:47.428614 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.428489    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-maas-api-tls\") pod \"maas-api-8669f899bf-59f8m\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:47.428614 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.428509    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xvdt\" (UniqueName: \"kubernetes.io/projected/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-kube-api-access-8xvdt\") pod \"maas-api-8669f899bf-59f8m\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:47.428614 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.428526    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-metrics-certs\") pod \"maas-api-8669f899bf-59f8m\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:47.429534 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.429507    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-openshift-service-ca\") pod \"maas-api-8669f899bf-59f8m\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:47.431193 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.431167    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-maas-api-tls\") pod \"maas-api-8669f899bf-59f8m\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:47.431272 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.431216    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-metrics-certs\") pod \"maas-api-8669f899bf-59f8m\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:47.489260 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.489232    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xvdt\" (UniqueName: \"kubernetes.io/projected/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-kube-api-access-8xvdt\") pod \"maas-api-8669f899bf-59f8m\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") " pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:47.520647 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.520601    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:47.893901 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:47.893873    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"]
Nov 03 20:46:47.896488 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:46:47.896459    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd5f959_4f8f_4f7b_a394_be2ad7d1eecb.slice/crio-fc0a5dfa34f7307d2a0c8610f7dc5fc6d55245e297d0cd5fb898826b90da2e45 WatchSource:0}: Error finding container fc0a5dfa34f7307d2a0c8610f7dc5fc6d55245e297d0cd5fb898826b90da2e45: Status 404 returned error can't find the container with id fc0a5dfa34f7307d2a0c8610f7dc5fc6d55245e297d0cd5fb898826b90da2e45
Nov 03 20:46:48.221362 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:48.221330    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m" event={"ID":"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb","Type":"ContainerStarted","Data":"fc0a5dfa34f7307d2a0c8610f7dc5fc6d55245e297d0cd5fb898826b90da2e45"}
Nov 03 20:46:49.227305 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:49.227270    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m" event={"ID":"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb","Type":"ContainerStarted","Data":"4c4e1ff46bad4f22cccefae49dc83d9ccc7234685d653c51719c68de3756e362"}
Nov 03 20:46:49.227691 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:49.227348    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:49.345745 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:49.345686    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m" podStartSLOduration=1.4984325809999999 podStartE2EDuration="2.345668058s" totalImagesPullingTime="847.235477ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:46:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:46:49.345198557 +0000 UTC m=+411.889904945" watchObservedRunningTime="2026-11-03 20:46:49.345668058 +0000 UTC m=+411.890374429"
Nov 03 20:46:55.236835 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.236805    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 20:46:55.406153 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.406124    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"]
Nov 03 20:46:55.406341 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.406317    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7" podUID="3966dfa8-e2af-4ff5-a076-15835ca00fca" containerName="maas-api" containerID="cri-o://9e42ce95d0096a12d43e444c3e2f6fc42faa87702769a4efcb65344e79a91b98" gracePeriod=30
Nov 03 20:46:55.654082 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.654053    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:55.693646 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.693601    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3966dfa8-e2af-4ff5-a076-15835ca00fca-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3966dfa8-e2af-4ff5-a076-15835ca00fca-maas-api-tls\") pod \"3966dfa8-e2af-4ff5-a076-15835ca00fca\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") "
Nov 03 20:46:55.693646 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.693647    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3966dfa8-e2af-4ff5-a076-15835ca00fca-kube-api-access-hm7bd\" (UniqueName: \"kubernetes.io/projected/3966dfa8-e2af-4ff5-a076-15835ca00fca-kube-api-access-hm7bd\") pod \"3966dfa8-e2af-4ff5-a076-15835ca00fca\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") "
Nov 03 20:46:55.693842 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.693665    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3966dfa8-e2af-4ff5-a076-15835ca00fca-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3966dfa8-e2af-4ff5-a076-15835ca00fca-metrics-certs\") pod \"3966dfa8-e2af-4ff5-a076-15835ca00fca\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") "
Nov 03 20:46:55.693842 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.693700    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3966dfa8-e2af-4ff5-a076-15835ca00fca-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3966dfa8-e2af-4ff5-a076-15835ca00fca-openshift-service-ca\") pod \"3966dfa8-e2af-4ff5-a076-15835ca00fca\" (UID: \"3966dfa8-e2af-4ff5-a076-15835ca00fca\") "
Nov 03 20:46:55.693994 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.693974    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3966dfa8-e2af-4ff5-a076-15835ca00fca-openshift-service-ca" pod "3966dfa8-e2af-4ff5-a076-15835ca00fca" (UID: "3966dfa8-e2af-4ff5-a076-15835ca00fca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 20:46:55.695970 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.695941    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3966dfa8-e2af-4ff5-a076-15835ca00fca-maas-api-tls" pod "3966dfa8-e2af-4ff5-a076-15835ca00fca" (UID: "3966dfa8-e2af-4ff5-a076-15835ca00fca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:46:55.695970 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.695958    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3966dfa8-e2af-4ff5-a076-15835ca00fca-kube-api-access-hm7bd" pod "3966dfa8-e2af-4ff5-a076-15835ca00fca" (UID: "3966dfa8-e2af-4ff5-a076-15835ca00fca"). InnerVolumeSpecName "kube-api-access-hm7bd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 20:46:55.696087 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.696033    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3966dfa8-e2af-4ff5-a076-15835ca00fca-metrics-certs" pod "3966dfa8-e2af-4ff5-a076-15835ca00fca" (UID: "3966dfa8-e2af-4ff5-a076-15835ca00fca"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 20:46:55.795044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.794981    4993 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3966dfa8-e2af-4ff5-a076-15835ca00fca-maas-api-tls\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:46:55.795044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.795005    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm7bd\" (UniqueName: \"kubernetes.io/projected/3966dfa8-e2af-4ff5-a076-15835ca00fca-kube-api-access-hm7bd\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:46:55.795044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.795014    4993 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3966dfa8-e2af-4ff5-a076-15835ca00fca-metrics-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:46:55.795044 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:55.795025    4993 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3966dfa8-e2af-4ff5-a076-15835ca00fca-openshift-service-ca\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 20:46:56.257962 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:56.257932    4993 generic.go:356] "Generic (PLEG): container finished" podID="3966dfa8-e2af-4ff5-a076-15835ca00fca" containerID="9e42ce95d0096a12d43e444c3e2f6fc42faa87702769a4efcb65344e79a91b98" exitCode=0
Nov 03 20:46:56.258443 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:56.257980    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"
Nov 03 20:46:56.258443 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:56.258022    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7" event={"ID":"3966dfa8-e2af-4ff5-a076-15835ca00fca","Type":"ContainerDied","Data":"9e42ce95d0096a12d43e444c3e2f6fc42faa87702769a4efcb65344e79a91b98"}
Nov 03 20:46:56.258443 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:56.258097    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7" event={"ID":"3966dfa8-e2af-4ff5-a076-15835ca00fca","Type":"ContainerDied","Data":"9b610b55e70ea3425cc17024e37dae83de92f5bbb14a5bbbb7a85d0be8a8a24b"}
Nov 03 20:46:56.258443 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:56.258133    4993 scope.go:122] "RemoveContainer" containerID="9e42ce95d0096a12d43e444c3e2f6fc42faa87702769a4efcb65344e79a91b98"
Nov 03 20:46:56.267340 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:56.267323    4993 scope.go:122] "RemoveContainer" containerID="9e42ce95d0096a12d43e444c3e2f6fc42faa87702769a4efcb65344e79a91b98"
Nov 03 20:46:56.267591 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:46:56.267572    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e42ce95d0096a12d43e444c3e2f6fc42faa87702769a4efcb65344e79a91b98\": container with ID starting with 9e42ce95d0096a12d43e444c3e2f6fc42faa87702769a4efcb65344e79a91b98 not found: ID does not exist" containerID="9e42ce95d0096a12d43e444c3e2f6fc42faa87702769a4efcb65344e79a91b98"
Nov 03 20:46:56.267652 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:56.267601    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e42ce95d0096a12d43e444c3e2f6fc42faa87702769a4efcb65344e79a91b98"} err="failed to get container status \"9e42ce95d0096a12d43e444c3e2f6fc42faa87702769a4efcb65344e79a91b98\": rpc error: code = NotFound desc = could not find container \"9e42ce95d0096a12d43e444c3e2f6fc42faa87702769a4efcb65344e79a91b98\": container with ID starting with 9e42ce95d0096a12d43e444c3e2f6fc42faa87702769a4efcb65344e79a91b98 not found: ID does not exist"
Nov 03 20:46:56.420397 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:56.420372    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"]
Nov 03 20:46:56.469996 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:56.469964    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-krcb7"]
Nov 03 20:46:57.575210 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:46:57.575174    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3966dfa8-e2af-4ff5-a076-15835ca00fca" path="/var/lib/kubelet/pods/3966dfa8-e2af-4ff5-a076-15835ca00fca/volumes"
Nov 03 20:47:01.669675 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:47:01.669616    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:47:31.668847 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:47:31.668810    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:47:52.373958 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:52.373880    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-658lq"]
Nov 03 20:47:52.374392 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:52.374236    4993 cpu_manager.go:414] "containerMap: removing container" podUID="3966dfa8-e2af-4ff5-a076-15835ca00fca" containerName="maas-api"
Nov 03 20:47:52.374392 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:52.374248    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3966dfa8-e2af-4ff5-a076-15835ca00fca" containerName="maas-api"
Nov 03 20:47:52.374392 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:52.374326    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="3966dfa8-e2af-4ff5-a076-15835ca00fca" containerName="maas-api"
Nov 03 20:47:52.377386 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:52.377368    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-658lq"
Nov 03 20:47:52.413165 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:52.413132    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq6tk\" (UniqueName: \"kubernetes.io/projected/b2744188-2043-4fd3-85c1-ed75eba9aae2-kube-api-access-gq6tk\") pod \"llmisvc-controller-manager-6466c554d8-658lq\" (UID: \"b2744188-2043-4fd3-85c1-ed75eba9aae2\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-658lq"
Nov 03 20:47:52.413276 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:52.413241    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2744188-2043-4fd3-85c1-ed75eba9aae2-cert\") pod \"llmisvc-controller-manager-6466c554d8-658lq\" (UID: \"b2744188-2043-4fd3-85c1-ed75eba9aae2\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-658lq"
Nov 03 20:47:52.479309 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:52.479287    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-jcv6l\""
Nov 03 20:47:52.479423 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:52.479320    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Nov 03 20:47:52.480490 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:52.480471    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-658lq"]
Nov 03 20:47:52.514170 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:52.514144    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq6tk\" (UniqueName: \"kubernetes.io/projected/b2744188-2043-4fd3-85c1-ed75eba9aae2-kube-api-access-gq6tk\") pod \"llmisvc-controller-manager-6466c554d8-658lq\" (UID: \"b2744188-2043-4fd3-85c1-ed75eba9aae2\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-658lq"
Nov 03 20:47:52.514276 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:52.514228    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2744188-2043-4fd3-85c1-ed75eba9aae2-cert\") pod \"llmisvc-controller-manager-6466c554d8-658lq\" (UID: \"b2744188-2043-4fd3-85c1-ed75eba9aae2\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-658lq"
Nov 03 20:47:52.516664 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:52.516644    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b2744188-2043-4fd3-85c1-ed75eba9aae2-cert\") pod \"llmisvc-controller-manager-6466c554d8-658lq\" (UID: \"b2744188-2043-4fd3-85c1-ed75eba9aae2\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-658lq"
Nov 03 20:47:52.575040 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:52.575022    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq6tk\" (UniqueName: \"kubernetes.io/projected/b2744188-2043-4fd3-85c1-ed75eba9aae2-kube-api-access-gq6tk\") pod \"llmisvc-controller-manager-6466c554d8-658lq\" (UID: \"b2744188-2043-4fd3-85c1-ed75eba9aae2\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-658lq"
Nov 03 20:47:52.689932 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:52.689875    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-658lq"
Nov 03 20:47:53.066564 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:53.066419    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-658lq"]
Nov 03 20:47:53.069229 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:47:53.069196    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2744188_2043_4fd3_85c1_ed75eba9aae2.slice/crio-8ccf47bb741fcfaadc43b728e360e9fb865333479aae8ad83854b1dd18c4a423 WatchSource:0}: Error finding container 8ccf47bb741fcfaadc43b728e360e9fb865333479aae8ad83854b1dd18c4a423: Status 404 returned error can't find the container with id 8ccf47bb741fcfaadc43b728e360e9fb865333479aae8ad83854b1dd18c4a423
Nov 03 20:47:53.496907 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:53.496871    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-658lq" event={"ID":"b2744188-2043-4fd3-85c1-ed75eba9aae2","Type":"ContainerStarted","Data":"8ccf47bb741fcfaadc43b728e360e9fb865333479aae8ad83854b1dd18c4a423"}
Nov 03 20:47:56.512404 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:56.512371    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-658lq" event={"ID":"b2744188-2043-4fd3-85c1-ed75eba9aae2","Type":"ContainerStarted","Data":"1ff8bf9b51c65f4b7805122f9b1a7e53bd12c0d0b320873f967098d3ab639e70"}
Nov 03 20:47:56.512807 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:56.512518    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-658lq"
Nov 03 20:47:56.629077 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:47:56.629020    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-658lq" podStartSLOduration=1.787794549 podStartE2EDuration="4.629003137s" totalImagesPullingTime="2.841208588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:47:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:47:56.627785757 +0000 UTC m=+479.172492152" watchObservedRunningTime="2026-11-03 20:47:56.629003137 +0000 UTC m=+479.173709507"
Nov 03 20:48:01.670555 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:48:01.670518    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:48:27.519178 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:48:27.519150    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-658lq"
Nov 03 20:48:31.669679 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:48:31.669640    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:49:01.669084 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:49:01.669047    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:49:31.368617 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.368569    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"]
Nov 03 20:49:31.372094 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.372070    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.424276 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.424242    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mmsql\""
Nov 03 20:49:31.454539 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.454513    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3861b24-aee3-4389-b01e-325fa453a7ce-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.454664 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.454545    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.454664 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.454571    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.454664 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.454587    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9l7b\" (UniqueName: \"kubernetes.io/projected/c3861b24-aee3-4389-b01e-325fa453a7ce-kube-api-access-n9l7b\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.454823 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.454730    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.454823 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.454755    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.454918 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.454818    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.454918 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.454846    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.454918 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.454910    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.473842 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.473819    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 03 20:49:31.473842 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.473833    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 03 20:49:31.473985 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.473900    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 03 20:49:31.514718 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.513606    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"]
Nov 03 20:49:31.555548 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.555525    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.555682 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.555557    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.555682 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.555577    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.555805 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.555711    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3861b24-aee3-4389-b01e-325fa453a7ce-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.555805 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.555762    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.555914 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.555803    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.555914 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.555828    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9l7b\" (UniqueName: \"kubernetes.io/projected/c3861b24-aee3-4389-b01e-325fa453a7ce-kube-api-access-n9l7b\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.555914 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.555883    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.556070 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.555911    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.556070 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.555951    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.556070 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.555997    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.556282 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.556258    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.556484 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.556455    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3861b24-aee3-4389-b01e-325fa453a7ce-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.556781 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.556591    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.558243 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.558221    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.558479 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.558463    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.616891 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.616866    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.617129 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.617108    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9l7b\" (UniqueName: \"kubernetes.io/projected/c3861b24-aee3-4389-b01e-325fa453a7ce-kube-api-access-n9l7b\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-slbzs\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:31.669502 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:49:31.669390    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:49:31.684511 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:31.684486    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:32.117509 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:32.117476    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"]
Nov 03 20:49:32.120505 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:49:32.120480    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3861b24_aee3_4389_b01e_325fa453a7ce.slice/crio-f0eb6f6ee258873ace71e96dbc36d145b654bd5a52ef52a38ac7b1b28da018a5 WatchSource:0}: Error finding container f0eb6f6ee258873ace71e96dbc36d145b654bd5a52ef52a38ac7b1b28da018a5: Status 404 returned error can't find the container with id f0eb6f6ee258873ace71e96dbc36d145b654bd5a52ef52a38ac7b1b28da018a5
Nov 03 20:49:32.900439 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:32.900407    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs" event={"ID":"c3861b24-aee3-4389-b01e-325fa453a7ce","Type":"ContainerStarted","Data":"f0eb6f6ee258873ace71e96dbc36d145b654bd5a52ef52a38ac7b1b28da018a5"}
Nov 03 20:49:36.034313 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:36.034281    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:49:36.034615 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:36.034342    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:49:36.034615 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:36.034369    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:49:36.923435 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:36.923396    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs" event={"ID":"c3861b24-aee3-4389-b01e-325fa453a7ce","Type":"ContainerStarted","Data":"b1f78712c6f40d69424c787e8bb7787c3900b2ac86d656e6260db959a27736a0"}
Nov 03 20:49:37.684947 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:37.684913    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:37.687653 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:37.686930    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 03 20:49:37.687653 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:37.686995    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs" podUID="c3861b24-aee3-4389-b01e-325fa453a7ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 03 20:49:38.688052 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:38.688016    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 03 20:49:38.688444 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:38.688077    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs" podUID="c3861b24-aee3-4389-b01e-325fa453a7ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 03 20:49:39.685998 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:39.685027    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 03 20:49:39.685998 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:39.685099    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs" podUID="c3861b24-aee3-4389-b01e-325fa453a7ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 03 20:49:40.686380 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:40.685963    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 03 20:49:40.686380 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:40.686026    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs" podUID="c3861b24-aee3-4389-b01e-325fa453a7ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 03 20:49:41.685016 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:41.684982    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 03 20:49:41.685259 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:41.685232    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs" podUID="c3861b24-aee3-4389-b01e-325fa453a7ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 03 20:49:42.684960 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:42.684923    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body=
Nov 03 20:49:42.685339 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:42.684985    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs" podUID="c3861b24-aee3-4389-b01e-325fa453a7ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused"
Nov 03 20:49:44.140041 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:44.140008    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:44.140468 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:44.140360    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:44.141283 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:44.141265    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 20:49:44.309881 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:49:44.309837    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs" podStartSLOduration=9.398713043 podStartE2EDuration="13.309825562s" totalImagesPullingTime="3.911112519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:49:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:49:37.151767128 +0000 UTC m=+579.696473498" watchObservedRunningTime="2026-11-03 20:49:44.309825562 +0000 UTC m=+586.854531922"
Nov 03 20:50:01.669207 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:50:01.669170    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:50:16.730106 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:16.730073    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"]
Nov 03 20:50:16.737791 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:16.737769    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:16.833229 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:16.833196    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"]
Nov 03 20:50:16.921833 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:16.921809    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bf4e14e-9896-452a-bfe9-58e3f89a155c-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:16.921924 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:16.921846    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bf4e14e-9896-452a-bfe9-58e3f89a155c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:16.921987 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:16.921946    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bf4e14e-9896-452a-bfe9-58e3f89a155c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:16.922036 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:16.922006    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf4e14e-9896-452a-bfe9-58e3f89a155c-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:16.922074 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:16.922050    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf4e14e-9896-452a-bfe9-58e3f89a155c-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:16.922113 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:16.922101    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bf4e14e-9896-452a-bfe9-58e3f89a155c-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:16.922149 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:16.922123    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spvh2\" (UniqueName: \"kubernetes.io/projected/6bf4e14e-9896-452a-bfe9-58e3f89a155c-kube-api-access-spvh2\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:16.922188 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:16.922148    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bf4e14e-9896-452a-bfe9-58e3f89a155c-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:16.922249 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:16.922228    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bf4e14e-9896-452a-bfe9-58e3f89a155c-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.022771 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.022708    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf4e14e-9896-452a-bfe9-58e3f89a155c-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.022771 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.022736    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bf4e14e-9896-452a-bfe9-58e3f89a155c-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.022771 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.022752    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spvh2\" (UniqueName: \"kubernetes.io/projected/6bf4e14e-9896-452a-bfe9-58e3f89a155c-kube-api-access-spvh2\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.023008 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.022776    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bf4e14e-9896-452a-bfe9-58e3f89a155c-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.023008 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.022857    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bf4e14e-9896-452a-bfe9-58e3f89a155c-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.023008 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.022906    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bf4e14e-9896-452a-bfe9-58e3f89a155c-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.023008 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.022931    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bf4e14e-9896-452a-bfe9-58e3f89a155c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.023008 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.022986    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bf4e14e-9896-452a-bfe9-58e3f89a155c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.023292 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.023026    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf4e14e-9896-452a-bfe9-58e3f89a155c-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.023292 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.023107    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf4e14e-9896-452a-bfe9-58e3f89a155c-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.023292 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.023274    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bf4e14e-9896-452a-bfe9-58e3f89a155c-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.023451 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.023342    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bf4e14e-9896-452a-bfe9-58e3f89a155c-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.023514 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.023469    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bf4e14e-9896-452a-bfe9-58e3f89a155c-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.023803 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.023782    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bf4e14e-9896-452a-bfe9-58e3f89a155c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.025061 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.025045    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bf4e14e-9896-452a-bfe9-58e3f89a155c-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.025418 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.025401    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bf4e14e-9896-452a-bfe9-58e3f89a155c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.085430 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.085403    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bf4e14e-9896-452a-bfe9-58e3f89a155c-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.085520 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.085469    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spvh2\" (UniqueName: \"kubernetes.io/projected/6bf4e14e-9896-452a-bfe9-58e3f89a155c-kube-api-access-spvh2\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-cws5p\" (UID: \"6bf4e14e-9896-452a-bfe9-58e3f89a155c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.351384 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.351325    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:17.776716 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.776690    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"]
Nov 03 20:50:17.779096 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:50:17.779064    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bf4e14e_9896_452a_bfe9_58e3f89a155c.slice/crio-db80a9e11060096f0efacb467f22aff8d03c47f4a0973a0ed99faa3182161241 WatchSource:0}: Error finding container db80a9e11060096f0efacb467f22aff8d03c47f4a0973a0ed99faa3182161241: Status 404 returned error can't find the container with id db80a9e11060096f0efacb467f22aff8d03c47f4a0973a0ed99faa3182161241
Nov 03 20:50:17.781363 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.781329    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:50:17.781440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.781396    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:50:17.781440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:17.781423    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:50:18.104549 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:18.104449    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p" event={"ID":"6bf4e14e-9896-452a-bfe9-58e3f89a155c","Type":"ContainerStarted","Data":"52c7d50849c28f2819f69fcdfdbc7ee90818931fc0dc2e813e28b05a95b0f053"}
Nov 03 20:50:18.104549 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:18.104509    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p" event={"ID":"6bf4e14e-9896-452a-bfe9-58e3f89a155c","Type":"ContainerStarted","Data":"db80a9e11060096f0efacb467f22aff8d03c47f4a0973a0ed99faa3182161241"}
Nov 03 20:50:18.225926 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:18.225870    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p" podStartSLOduration=2.225855137 podStartE2EDuration="2.225855137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:50:18.225074368 +0000 UTC m=+620.769780738" watchObservedRunningTime="2026-11-03 20:50:18.225855137 +0000 UTC m=+620.770561504"
Nov 03 20:50:18.352220 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:18.352188    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:18.353642 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:18.353607    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 03 20:50:18.353753 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:18.353666    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p" podUID="6bf4e14e-9896-452a-bfe9-58e3f89a155c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 03 20:50:19.352226 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:19.352197    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 03 20:50:19.352595 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:19.352249    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p" podUID="6bf4e14e-9896-452a-bfe9-58e3f89a155c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 03 20:50:20.352011 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:20.351976    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 03 20:50:20.352231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:20.352052    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p" podUID="6bf4e14e-9896-452a-bfe9-58e3f89a155c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 03 20:50:21.351865 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:21.351834    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 03 20:50:21.352308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:21.351896    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p" podUID="6bf4e14e-9896-452a-bfe9-58e3f89a155c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 03 20:50:22.352170 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:22.352138    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 03 20:50:22.352546 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:22.352196    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p" podUID="6bf4e14e-9896-452a-bfe9-58e3f89a155c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 03 20:50:23.351831 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:23.351800    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 03 20:50:23.352004 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:23.351852    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p" podUID="6bf4e14e-9896-452a-bfe9-58e3f89a155c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 03 20:50:24.352186 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:24.352148    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body=
Nov 03 20:50:24.352683 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:24.352209    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p" podUID="6bf4e14e-9896-452a-bfe9-58e3f89a155c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused"
Nov 03 20:50:25.952794 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:25.952766    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:25.953518 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:25.953410    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:25.954345 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:50:25.954326    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p"
Nov 03 20:50:31.669093 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:50:31.669050    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:51:01.670294 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.670248    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"]
Nov 03 20:51:01.674417 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.674397    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.675399 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:51:01.675371    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:51:01.726899 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.726863    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-499187-openshift-default-dockercfg-jsmm9\""
Nov 03 20:51:01.777664 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.777617    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"]
Nov 03 20:51:01.814290 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.814254    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.814487 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.814306    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.814487 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.814344    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be1af043-dda6-4590-8f69-36c2abfcd868-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.814487 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.814455    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xvbd\" (UniqueName: \"kubernetes.io/projected/be1af043-dda6-4590-8f69-36c2abfcd868-kube-api-access-9xvbd\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.814703 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.814508    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.814703 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.814612    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be1af043-dda6-4590-8f69-36c2abfcd868-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.814789 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.814728    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be1af043-dda6-4590-8f69-36c2abfcd868-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.814842 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.814799    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.814899 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.814878    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.915331 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.915301    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be1af043-dda6-4590-8f69-36c2abfcd868-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.915331 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.915336    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be1af043-dda6-4590-8f69-36c2abfcd868-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.915585 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.915370    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.915585 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.915413    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.915585 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.915450    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.915585 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.915475    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.915585 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.915507    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be1af043-dda6-4590-8f69-36c2abfcd868-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.915585 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.915548    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xvbd\" (UniqueName: \"kubernetes.io/projected/be1af043-dda6-4590-8f69-36c2abfcd868-kube-api-access-9xvbd\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.915585 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.915586    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.915995 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.915918    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.916066 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.916036    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.916197 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.916173    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.916313 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.916206    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.917003 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.916976    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be1af043-dda6-4590-8f69-36c2abfcd868-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.918234 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.918210    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.918712 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.918690    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be1af043-dda6-4590-8f69-36c2abfcd868-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.976882 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.976850    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xvbd\" (UniqueName: \"kubernetes.io/projected/be1af043-dda6-4590-8f69-36c2abfcd868-kube-api-access-9xvbd\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.977040 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.977022    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be1af043-dda6-4590-8f69-36c2abfcd868-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:01.990781 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:01.990760    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:02.002065 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.002037    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"]
Nov 03 20:51:02.006008 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.005992    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.058507 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.058472    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-e22abf-openshift-default-dockercfg-8q8f9\""
Nov 03 20:51:02.065697 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.065668    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"]
Nov 03 20:51:02.118291 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.118253    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.118465 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.118299    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.118465 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.118411    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.118465 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.118440    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96fbe5e3-4665-45ac-bc64-f128c312fb05-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.118667 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.118467    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.118667 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.118482    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.118667 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.118543    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmnj6\" (UniqueName: \"kubernetes.io/projected/96fbe5e3-4665-45ac-bc64-f128c312fb05-kube-api-access-dmnj6\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.118813 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.118674    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.118813 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.118713    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.219861 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.219815    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.219861 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.219850    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.220111 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.219880    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.220111 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.219995    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96fbe5e3-4665-45ac-bc64-f128c312fb05-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.220111 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.220060    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.220111 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.220088    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.220327 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.220118    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmnj6\" (UniqueName: \"kubernetes.io/projected/96fbe5e3-4665-45ac-bc64-f128c312fb05-kube-api-access-dmnj6\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.220327 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.220152    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.220327 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.220187    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.220486 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.220402    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.220573 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.220545    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.220696 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.220675    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.220764 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.220710    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.220823 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.220786    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96fbe5e3-4665-45ac-bc64-f128c312fb05-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.222308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.222286    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.222696 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.222679    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.280499 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.280431    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.280655 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.280552    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmnj6\" (UniqueName: \"kubernetes.io/projected/96fbe5e3-4665-45ac-bc64-f128c312fb05-kube-api-access-dmnj6\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.327411 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.327364    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:02.437732 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.437702    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"]
Nov 03 20:51:02.443166 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:51:02.443123    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe1af043_dda6_4590_8f69_36c2abfcd868.slice/crio-7c14f3bc6209587b2943409d7b0f8362e74eacaf59e52d3e8ab44506dd910715 WatchSource:0}: Error finding container 7c14f3bc6209587b2943409d7b0f8362e74eacaf59e52d3e8ab44506dd910715: Status 404 returned error can't find the container with id 7c14f3bc6209587b2943409d7b0f8362e74eacaf59e52d3e8ab44506dd910715
Nov 03 20:51:02.446231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.446195    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:02.446326 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.446269    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:02.446326 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.446298    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:02.759559 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.759519    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"]
Nov 03 20:51:02.771593 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:51:02.771565    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96fbe5e3_4665_45ac_bc64_f128c312fb05.slice/crio-2994421c913b40fa9acfcceda9ecff5825168a15f44026dbc4da44a3d339e675 WatchSource:0}: Error finding container 2994421c913b40fa9acfcceda9ecff5825168a15f44026dbc4da44a3d339e675: Status 404 returned error can't find the container with id 2994421c913b40fa9acfcceda9ecff5825168a15f44026dbc4da44a3d339e675
Nov 03 20:51:02.774664 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.774604    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:02.774767 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.774701    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:02.774767 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:02.774732    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:03.305031 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:03.304991    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552" event={"ID":"be1af043-dda6-4590-8f69-36c2abfcd868","Type":"ContainerStarted","Data":"bed8089f07c51ef3556204d691ccb9e523641ed071daa260b8c810b30f06cf89"}
Nov 03 20:51:03.305228 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:03.305060    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552" event={"ID":"be1af043-dda6-4590-8f69-36c2abfcd868","Type":"ContainerStarted","Data":"7c14f3bc6209587b2943409d7b0f8362e74eacaf59e52d3e8ab44506dd910715"}
Nov 03 20:51:03.306765 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:03.306736    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79" event={"ID":"96fbe5e3-4665-45ac-bc64-f128c312fb05","Type":"ContainerStarted","Data":"056409b81300dc4b86f1042405019bd6a73cdb77a134d8d96c2334a89d63ebcb"}
Nov 03 20:51:03.306877 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:03.306792    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79" event={"ID":"96fbe5e3-4665-45ac-bc64-f128c312fb05","Type":"ContainerStarted","Data":"2994421c913b40fa9acfcceda9ecff5825168a15f44026dbc4da44a3d339e675"}
Nov 03 20:51:03.327847 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:03.327817    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:03.333862 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:03.333835    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:03.426179 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:03.426120    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552" podStartSLOduration=2.426103208 podStartE2EDuration="2.426103208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:51:03.425826826 +0000 UTC m=+665.970533213" watchObservedRunningTime="2026-11-03 20:51:03.426103208 +0000 UTC m=+665.970809577"
Nov 03 20:51:03.548533 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:03.548466    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79" podStartSLOduration=2.548445065 podStartE2EDuration="2.548445065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:51:03.545807367 +0000 UTC m=+666.090513775" watchObservedRunningTime="2026-11-03 20:51:03.548445065 +0000 UTC m=+666.093151437"
Nov 03 20:51:03.991418 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:03.991381    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:03.996715 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:03.996682    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:04.311383 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:04.311291    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:04.311383 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:04.311346    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:04.312172 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:04.312152    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:04.312276 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:04.312211    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:11.776581 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:11.776540    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"]
Nov 03 20:51:11.780093 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:11.780070    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:11.832046 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:11.832007    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-12a423-serving-cert\""
Nov 03 20:51:11.882922 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:11.882882    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"]
Nov 03 20:51:11.909825 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:11.909789    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:11.910026 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:11.909854    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-maas-api-tls\") pod \"maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:11.910026 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:11.909961    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqqfg\" (UniqueName: \"kubernetes.io/projected/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-kube-api-access-fqqfg\") pod \"maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:11.910026 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:11.910004    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-metrics-certs\") pod \"maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:12.011156 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:12.011114    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:12.011156 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:12.011161    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-maas-api-tls\") pod \"maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:12.011399 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:12.011200    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqqfg\" (UniqueName: \"kubernetes.io/projected/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-kube-api-access-fqqfg\") pod \"maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:12.011399 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:12.011222    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-metrics-certs\") pod \"maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:12.011515 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:51:12.011440    4993 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-serving-cert: secret "maas-api-e2e-worker-w6-12a423-serving-cert" not found
Nov 03 20:51:12.011565 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:51:12.011535    4993 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-maas-api-tls podName:85847bd0-1857-4ba7-b4c4-ada49fc94ae3 nodeName:}" failed. No retries permitted until 2026-11-03 20:51:12.51151142 +0000 UTC m=+675.056217784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-maas-api-tls") pod "maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm" (UID: "85847bd0-1857-4ba7-b4c4-ada49fc94ae3") : secret "maas-api-e2e-worker-w6-12a423-serving-cert" not found
Nov 03 20:51:12.012041 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:12.012017    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:12.014023 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:12.013997    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-metrics-certs\") pod \"maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:12.073102 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:12.073028    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqqfg\" (UniqueName: \"kubernetes.io/projected/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-kube-api-access-fqqfg\") pod \"maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:12.515987 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:12.515950    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-maas-api-tls\") pod \"maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:12.518620 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:12.518590    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-maas-api-tls\") pod \"maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:12.693776 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:12.693743    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:13.069098 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:13.069074    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"]
Nov 03 20:51:13.071717 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:51:13.071690    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85847bd0_1857_4ba7_b4c4_ada49fc94ae3.slice/crio-a8a8e4353ba8e48007ed622c44c48e6d548adab149959bb27b607764218821cd WatchSource:0}: Error finding container a8a8e4353ba8e48007ed622c44c48e6d548adab149959bb27b607764218821cd: Status 404 returned error can't find the container with id a8a8e4353ba8e48007ed622c44c48e6d548adab149959bb27b607764218821cd
Nov 03 20:51:13.073115 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:13.073094    4993 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 20:51:13.353578 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:13.353489    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm" event={"ID":"85847bd0-1857-4ba7-b4c4-ada49fc94ae3","Type":"ContainerStarted","Data":"a8a8e4353ba8e48007ed622c44c48e6d548adab149959bb27b607764218821cd"}
Nov 03 20:51:14.359826 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:14.359734    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm" event={"ID":"85847bd0-1857-4ba7-b4c4-ada49fc94ae3","Type":"ContainerStarted","Data":"335e311dcc7b9c609407b6820499a740b4e04fc161130baa3670cf8b6ce7257d"}
Nov 03 20:51:14.360193 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:14.359874    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:17.118746 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.118685    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm" podStartSLOduration=5.193866965 podStartE2EDuration="6.118669824s" totalImagesPullingTime="924.802859ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:51:14.491095216 +0000 UTC m=+677.035801586" watchObservedRunningTime="2026-11-03 20:51:17.118669824 +0000 UTC m=+679.663376202"
Nov 03 20:51:17.119237 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.118843    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"]
Nov 03 20:51:17.122521 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.122493    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:17.156989 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.156957    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-maas-api-tls\") pod \"maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:17.156989 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.156991    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-metrics-certs\") pod \"maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:17.157155 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.157102    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7bsz\" (UniqueName: \"kubernetes.io/projected/b380cf76-c519-4719-84fa-5c3025ea84f1-kube-api-access-f7bsz\") pod \"maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:17.157192 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.157164    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b380cf76-c519-4719-84fa-5c3025ea84f1-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:17.174517 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.174458    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-89bb58-serving-cert\""
Nov 03 20:51:17.225434 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.225400    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"]
Nov 03 20:51:17.257652 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.257590    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-maas-api-tls\") pod \"maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:17.257652 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.257646    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-metrics-certs\") pod \"maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:17.257895 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.257708    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7bsz\" (UniqueName: \"kubernetes.io/projected/b380cf76-c519-4719-84fa-5c3025ea84f1-kube-api-access-f7bsz\") pod \"maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:17.257895 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:51:17.257753    4993 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-serving-cert: secret "maas-api-e2e-worker-w3-89bb58-serving-cert" not found
Nov 03 20:51:17.257895 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.257781    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b380cf76-c519-4719-84fa-5c3025ea84f1-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:17.257895 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:51:17.257832    4993 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-maas-api-tls podName:b380cf76-c519-4719-84fa-5c3025ea84f1 nodeName:}" failed. No retries permitted until 2026-11-03 20:51:17.757811694 +0000 UTC m=+680.302518049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-maas-api-tls") pod "maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv" (UID: "b380cf76-c519-4719-84fa-5c3025ea84f1") : secret "maas-api-e2e-worker-w3-89bb58-serving-cert" not found
Nov 03 20:51:17.258601 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.258579    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b380cf76-c519-4719-84fa-5c3025ea84f1-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:17.260214 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.260194    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-metrics-certs\") pod \"maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:17.318943 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.318915    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7bsz\" (UniqueName: \"kubernetes.io/projected/b380cf76-c519-4719-84fa-5c3025ea84f1-kube-api-access-f7bsz\") pod \"maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:17.761687 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.761620    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-maas-api-tls\") pod \"maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:17.764132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:17.764105    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-maas-api-tls\") pod \"maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:18.035666 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:18.035572    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:18.412057 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:18.412033    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"]
Nov 03 20:51:18.414065 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:51:18.414039    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb380cf76_c519_4719_84fa_5c3025ea84f1.slice/crio-065160dfec5005be4862388905a78cabc45d96903ec509c0bd92275296fd7f24 WatchSource:0}: Error finding container 065160dfec5005be4862388905a78cabc45d96903ec509c0bd92275296fd7f24: Status 404 returned error can't find the container with id 065160dfec5005be4862388905a78cabc45d96903ec509c0bd92275296fd7f24
Nov 03 20:51:19.388165 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:19.388128    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv" event={"ID":"b380cf76-c519-4719-84fa-5c3025ea84f1","Type":"ContainerStarted","Data":"6f1b2bcab3ed951a0e54adc78b2c60cac813eeab3f70773a0a0cc0b01be902e4"}
Nov 03 20:51:19.388342 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:19.388180    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv" event={"ID":"b380cf76-c519-4719-84fa-5c3025ea84f1","Type":"ContainerStarted","Data":"065160dfec5005be4862388905a78cabc45d96903ec509c0bd92275296fd7f24"}
Nov 03 20:51:19.388342 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:19.388264    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:19.533251 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:19.533144    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv" podStartSLOduration=1.691663754 podStartE2EDuration="2.533128343s" totalImagesPullingTime="841.464589ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:51:19.532766159 +0000 UTC m=+682.077472530" watchObservedRunningTime="2026-11-03 20:51:19.533128343 +0000 UTC m=+682.077834715"
Nov 03 20:51:20.370239 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:20.370214    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 20:51:22.491223 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:22.491188    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"]
Nov 03 20:51:22.494870 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:22.494852    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:22.499260 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:22.499241    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-metrics-certs\") pod \"maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:22.499365 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:22.499282    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcdc5f37-f37d-43bc-8fea-ad101a050eec-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:22.499365 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:22.499310    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92czx\" (UniqueName: \"kubernetes.io/projected/dcdc5f37-f37d-43bc-8fea-ad101a050eec-kube-api-access-92czx\") pod \"maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:22.499459 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:22.499409    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-maas-api-tls\") pod \"maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:22.548831 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:22.548800    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-d01cef-serving-cert\""
Nov 03 20:51:22.599829 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:22.599798    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-metrics-certs\") pod \"maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:22.599980 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:22.599852    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcdc5f37-f37d-43bc-8fea-ad101a050eec-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:22.599980 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:22.599966    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92czx\" (UniqueName: \"kubernetes.io/projected/dcdc5f37-f37d-43bc-8fea-ad101a050eec-kube-api-access-92czx\") pod \"maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:22.600078 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:22.600008    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-maas-api-tls\") pod \"maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:22.600078 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:22.600043    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"]
Nov 03 20:51:22.600188 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:51:22.600161    4993 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-serving-cert: secret "maas-api-e2e-worker-w1-d01cef-serving-cert" not found
Nov 03 20:51:22.600249 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:51:22.600239    4993 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-maas-api-tls podName:dcdc5f37-f37d-43bc-8fea-ad101a050eec nodeName:}" failed. No retries permitted until 2026-11-03 20:51:23.100219156 +0000 UTC m=+685.644925518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-maas-api-tls") pod "maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg" (UID: "dcdc5f37-f37d-43bc-8fea-ad101a050eec") : secret "maas-api-e2e-worker-w1-d01cef-serving-cert" not found
Nov 03 20:51:22.600483 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:22.600463    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcdc5f37-f37d-43bc-8fea-ad101a050eec-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:22.602457 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:22.602437    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-metrics-certs\") pod \"maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:22.661891 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:22.661862    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92czx\" (UniqueName: \"kubernetes.io/projected/dcdc5f37-f37d-43bc-8fea-ad101a050eec-kube-api-access-92czx\") pod \"maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:23.104247 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:23.104213    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-maas-api-tls\") pod \"maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:23.107062 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:23.107032    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-maas-api-tls\") pod \"maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:23.107221 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:23.107172    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:23.486154 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:23.486108    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"]
Nov 03 20:51:23.488291 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:51:23.488266    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcdc5f37_f37d_43bc_8fea_ad101a050eec.slice/crio-a0f60d2f9177077d446b4699a9fc7dcb3c1129c850a106c4cd5ea5c0608fd864 WatchSource:0}: Error finding container a0f60d2f9177077d446b4699a9fc7dcb3c1129c850a106c4cd5ea5c0608fd864: Status 404 returned error can't find the container with id a0f60d2f9177077d446b4699a9fc7dcb3c1129c850a106c4cd5ea5c0608fd864
Nov 03 20:51:24.414068 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:24.414032    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg" event={"ID":"dcdc5f37-f37d-43bc-8fea-ad101a050eec","Type":"ContainerStarted","Data":"b3b2cecabb8048e7170d8c895f0008dfe4276694ceb99c768a2c0a66970c9f79"}
Nov 03 20:51:24.414416 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:24.414084    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg" event={"ID":"dcdc5f37-f37d-43bc-8fea-ad101a050eec","Type":"ContainerStarted","Data":"a0f60d2f9177077d446b4699a9fc7dcb3c1129c850a106c4cd5ea5c0608fd864"}
Nov 03 20:51:24.414416 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:24.414169    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:24.574157 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:24.574052    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg" podStartSLOduration=1.745205082 podStartE2EDuration="2.574036725s" totalImagesPullingTime="828.831643ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:51:24.573343719 +0000 UTC m=+687.118050094" watchObservedRunningTime="2026-11-03 20:51:24.574036725 +0000 UTC m=+687.118743095"
Nov 03 20:51:25.398582 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:25.398552    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 20:51:27.785529 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:27.785489    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"]
Nov 03 20:51:27.789129 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:27.789111    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:27.842288 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:27.842256    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-499187-serving-cert\""
Nov 03 20:51:27.850868 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:27.850836    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-maas-api-tls\") pod \"maas-api-e2e-worker-w0-499187-6454d55477-sf6x5\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:27.850999 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:27.850880    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fee816e-2cfa-4ba1-a6b1-a36260877599-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-499187-6454d55477-sf6x5\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:27.851034 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:27.851010    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd7d5\" (UniqueName: \"kubernetes.io/projected/3fee816e-2cfa-4ba1-a6b1-a36260877599-kube-api-access-dd7d5\") pod \"maas-api-e2e-worker-w0-499187-6454d55477-sf6x5\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:27.851173 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:27.851156    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-metrics-certs\") pod \"maas-api-e2e-worker-w0-499187-6454d55477-sf6x5\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:27.893563 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:27.893527    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"]
Nov 03 20:51:27.952364 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:27.952332    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-metrics-certs\") pod \"maas-api-e2e-worker-w0-499187-6454d55477-sf6x5\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:27.952543 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:27.952374    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-maas-api-tls\") pod \"maas-api-e2e-worker-w0-499187-6454d55477-sf6x5\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:27.952543 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:27.952394    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fee816e-2cfa-4ba1-a6b1-a36260877599-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-499187-6454d55477-sf6x5\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:27.952543 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:27.952431    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd7d5\" (UniqueName: \"kubernetes.io/projected/3fee816e-2cfa-4ba1-a6b1-a36260877599-kube-api-access-dd7d5\") pod \"maas-api-e2e-worker-w0-499187-6454d55477-sf6x5\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:27.952543 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:51:27.952485    4993 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-serving-cert: secret "maas-api-e2e-worker-w0-499187-serving-cert" not found
Nov 03 20:51:27.952786 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:51:27.952574    4993 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-maas-api-tls podName:3fee816e-2cfa-4ba1-a6b1-a36260877599 nodeName:}" failed. No retries permitted until 2026-11-03 20:51:28.452557589 +0000 UTC m=+690.997263941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-maas-api-tls") pod "maas-api-e2e-worker-w0-499187-6454d55477-sf6x5" (UID: "3fee816e-2cfa-4ba1-a6b1-a36260877599") : secret "maas-api-e2e-worker-w0-499187-serving-cert" not found
Nov 03 20:51:27.953166 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:27.953144    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fee816e-2cfa-4ba1-a6b1-a36260877599-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-499187-6454d55477-sf6x5\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:27.954884 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:27.954862    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-metrics-certs\") pod \"maas-api-e2e-worker-w0-499187-6454d55477-sf6x5\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:28.016167 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:28.016132    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd7d5\" (UniqueName: \"kubernetes.io/projected/3fee816e-2cfa-4ba1-a6b1-a36260877599-kube-api-access-dd7d5\") pod \"maas-api-e2e-worker-w0-499187-6454d55477-sf6x5\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:28.457980 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:28.457945    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-maas-api-tls\") pod \"maas-api-e2e-worker-w0-499187-6454d55477-sf6x5\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:28.460565 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:28.460541    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-maas-api-tls\") pod \"maas-api-e2e-worker-w0-499187-6454d55477-sf6x5\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:28.702278 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:28.702239    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:29.078469 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:29.078442    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"]
Nov 03 20:51:29.080575 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:51:29.080548    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fee816e_2cfa_4ba1_a6b1_a36260877599.slice/crio-cfd809bfaf7d2f854d9c4cf18dac34a5b052d0a30609cda05405a11f0643a814 WatchSource:0}: Error finding container cfd809bfaf7d2f854d9c4cf18dac34a5b052d0a30609cda05405a11f0643a814: Status 404 returned error can't find the container with id cfd809bfaf7d2f854d9c4cf18dac34a5b052d0a30609cda05405a11f0643a814
Nov 03 20:51:29.440182 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:29.440097    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5" event={"ID":"3fee816e-2cfa-4ba1-a6b1-a36260877599","Type":"ContainerStarted","Data":"cfd809bfaf7d2f854d9c4cf18dac34a5b052d0a30609cda05405a11f0643a814"}
Nov 03 20:51:30.425148 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:30.425066    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 20:51:30.448593 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:30.448554    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5" event={"ID":"3fee816e-2cfa-4ba1-a6b1-a36260877599","Type":"ContainerStarted","Data":"440eb38a1e4299fe1b3ab4ac256cb7d51999a855eb45a141527bcb8bca0df9a0"}
Nov 03 20:51:30.448812 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:30.448663    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:30.666446 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:30.666392    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5" podStartSLOduration=2.745844969 podStartE2EDuration="3.666378038s" totalImagesPullingTime="920.533069ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:51:30.665377402 +0000 UTC m=+693.210083773" watchObservedRunningTime="2026-11-03 20:51:30.666378038 +0000 UTC m=+693.211084408"
Nov 03 20:51:31.671164 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:51:31.671128    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:51:36.461601 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:36.461575    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 20:51:37.982576 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:37.982545    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"]
Nov 03 20:51:37.985765 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:37.985750    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:38.041534 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.041511    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-ca4642-serving-cert\""
Nov 03 20:51:38.049839 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.049809    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-maas-api-tls\") pod \"maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:38.049970 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.049846    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-metrics-certs\") pod \"maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:38.049970 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.049951    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:38.050084 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.049972    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw869\" (UniqueName: \"kubernetes.io/projected/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-kube-api-access-pw869\") pod \"maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:38.093128 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.093101    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"]
Nov 03 20:51:38.150500 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.150477    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:38.150643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.150511    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw869\" (UniqueName: \"kubernetes.io/projected/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-kube-api-access-pw869\") pod \"maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:38.150643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.150543    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-maas-api-tls\") pod \"maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:38.150643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.150563    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-metrics-certs\") pod \"maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:38.150814 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:51:38.150698    4993 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-serving-cert: secret "maas-api-e2e-worker-w4-ca4642-serving-cert" not found
Nov 03 20:51:38.150814 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:51:38.150771    4993 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-maas-api-tls podName:609e8c8e-04cc-4f27-b80a-0ab170d8ab57 nodeName:}" failed. No retries permitted until 2026-11-03 20:51:38.650750616 +0000 UTC m=+701.195456981 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-maas-api-tls") pod "maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz" (UID: "609e8c8e-04cc-4f27-b80a-0ab170d8ab57") : secret "maas-api-e2e-worker-w4-ca4642-serving-cert" not found
Nov 03 20:51:38.151167 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.151148    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:38.153182 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.153160    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-metrics-certs\") pod \"maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:38.212038 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.211998    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw869\" (UniqueName: \"kubernetes.io/projected/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-kube-api-access-pw869\") pod \"maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:38.654557 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.654523    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-maas-api-tls\") pod \"maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:38.657137 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.657113    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-maas-api-tls\") pod \"maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:38.898639 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:38.898609    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:39.276475 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:39.276451    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"]
Nov 03 20:51:39.279177 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:51:39.279154    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod609e8c8e_04cc_4f27_b80a_0ab170d8ab57.slice/crio-094f0aad73b1c18d3fd44ef584b139711a386acd2249d7898a6f2c2c2276669b WatchSource:0}: Error finding container 094f0aad73b1c18d3fd44ef584b139711a386acd2249d7898a6f2c2c2276669b: Status 404 returned error can't find the container with id 094f0aad73b1c18d3fd44ef584b139711a386acd2249d7898a6f2c2c2276669b
Nov 03 20:51:39.496642 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:39.496598    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz" event={"ID":"609e8c8e-04cc-4f27-b80a-0ab170d8ab57","Type":"ContainerStarted","Data":"094f0aad73b1c18d3fd44ef584b139711a386acd2249d7898a6f2c2c2276669b"}
Nov 03 20:51:40.503375 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:40.503337    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz" event={"ID":"609e8c8e-04cc-4f27-b80a-0ab170d8ab57","Type":"ContainerStarted","Data":"b9f1467779be4321b6a90fc6a22b6d54be0ac9479b7d408f8b19d7029bfe5de5"}
Nov 03 20:51:40.503760 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:40.503455    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:40.621058 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:40.621009    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz" podStartSLOduration=2.744258531 podStartE2EDuration="3.620994098s" totalImagesPullingTime="876.735567ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:51:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:51:40.62008681 +0000 UTC m=+703.164793181" watchObservedRunningTime="2026-11-03 20:51:40.620994098 +0000 UTC m=+703.165700459"
Nov 03 20:51:46.526658 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:46.525618    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 20:51:49.994987 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:49.994034    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": read tcp 10.133.0.2:43286->10.133.0.32:15021: read: connection reset by peer" start-of-body=
Nov 03 20:51:49.994987 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:49.994120    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79" podUID="96fbe5e3-4665-45ac-bc64-f128c312fb05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": read tcp 10.133.0.2:43286->10.133.0.32:15021: read: connection reset by peer"
Nov 03 20:51:50.352781 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:50.352744    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 20:51:50.353795 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:50.352868    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552" podUID="be1af043-dda6-4590-8f69-36c2abfcd868" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 20:51:50.562856 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:50.562781    4993 generic.go:356] "Generic (PLEG): container finished" podID="be1af043-dda6-4590-8f69-36c2abfcd868" containerID="bed8089f07c51ef3556204d691ccb9e523641ed071daa260b8c810b30f06cf89" exitCode=0
Nov 03 20:51:50.562998 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:50.562858    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552" event={"ID":"be1af043-dda6-4590-8f69-36c2abfcd868","Type":"ContainerDied","Data":"bed8089f07c51ef3556204d691ccb9e523641ed071daa260b8c810b30f06cf89"}
Nov 03 20:51:50.563314 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:50.563288    4993 scope.go:122] "RemoveContainer" containerID="bed8089f07c51ef3556204d691ccb9e523641ed071daa260b8c810b30f06cf89"
Nov 03 20:51:50.564701 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:50.564683    4993 generic.go:356] "Generic (PLEG): container finished" podID="96fbe5e3-4665-45ac-bc64-f128c312fb05" containerID="056409b81300dc4b86f1042405019bd6a73cdb77a134d8d96c2334a89d63ebcb" exitCode=0
Nov 03 20:51:50.564774 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:50.564710    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79" event={"ID":"96fbe5e3-4665-45ac-bc64-f128c312fb05","Type":"ContainerDied","Data":"056409b81300dc4b86f1042405019bd6a73cdb77a134d8d96c2334a89d63ebcb"}
Nov 03 20:51:50.564774 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:50.564702    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:50.564840 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:50.564804    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:50.564875 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:50.564841    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:50.564989 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:50.564977    4993 scope.go:122] "RemoveContainer" containerID="056409b81300dc4b86f1042405019bd6a73cdb77a134d8d96c2334a89d63ebcb"
Nov 03 20:51:50.566006 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:50.565983    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:50.566069 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:50.566042    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:50.566104 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:50.566072    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:51:51.616454 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:51.616418    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79" event={"ID":"96fbe5e3-4665-45ac-bc64-f128c312fb05","Type":"ContainerStarted","Data":"d30e47f236b33479dfd2e242fce60c5c09f7a576165a6629d9b8b231769ac323"}
Nov 03 20:51:51.618207 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:51.618180    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552" event={"ID":"be1af043-dda6-4590-8f69-36c2abfcd868","Type":"ContainerStarted","Data":"2bccd6f3c7127d7882de17ee864298e584a576e4c88c2b953eeefe38609e8121"}
Nov 03 20:51:51.998655 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:51.998034    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:51.998655 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:51.998163    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 03 20:51:51.998655 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:51.998206    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552" podUID="be1af043-dda6-4590-8f69-36c2abfcd868" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 03 20:51:52.328145 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:52.328038    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:52.328330 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:52.328259    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 03 20:51:52.328330 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:52.328310    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79" podUID="96fbe5e3-4665-45ac-bc64-f128c312fb05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 03 20:51:52.997170 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:52.997137    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 03 20:51:52.997841 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:52.997790    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552" podUID="be1af043-dda6-4590-8f69-36c2abfcd868" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 03 20:51:53.328611 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:53.328522    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 03 20:51:53.328611 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:53.328582    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79" podUID="96fbe5e3-4665-45ac-bc64-f128c312fb05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 03 20:51:53.999656 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:53.995964    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 03 20:51:53.999656 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:53.996024    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552" podUID="be1af043-dda6-4590-8f69-36c2abfcd868" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 03 20:51:54.328331 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:54.328242    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 03 20:51:54.328331 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:54.328303    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79" podUID="96fbe5e3-4665-45ac-bc64-f128c312fb05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 03 20:51:54.992068 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:54.992027    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 03 20:51:54.992812 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:54.992373    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552" podUID="be1af043-dda6-4590-8f69-36c2abfcd868" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 03 20:51:55.327960 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:55.327874    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 03 20:51:55.327960 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:55.327929    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79" podUID="96fbe5e3-4665-45ac-bc64-f128c312fb05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 03 20:51:55.994655 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:55.994043    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Nov 03 20:51:55.994655 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:55.994106    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552" podUID="be1af043-dda6-4590-8f69-36c2abfcd868" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Nov 03 20:51:56.328514 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:56.328429    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Nov 03 20:51:56.328514 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:56.328480    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79" podUID="96fbe5e3-4665-45ac-bc64-f128c312fb05" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Nov 03 20:51:56.996765 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:56.996015    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:56.996765 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:56.996714    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:56.997475 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:56.997421    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 20:51:57.332249 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:57.332171    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:57.332598 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:57.332515    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:51:57.333127 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:51:57.333108    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 20:52:00.696232 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.696200    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"]
Nov 03 20:52:00.699378 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.699358    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.784072 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.784042    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-12a423\"/\"kube-root-ca.crt\""
Nov 03 20:52:00.801136 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.801107    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-12a423\"/\"openshift-service-ca.crt\""
Nov 03 20:52:00.801369 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.801350    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-12a423\"/\"default-dockercfg-5rvzg\""
Nov 03 20:52:00.801427 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.801387    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-12a423\"/\"simulator-w6-12a423-kserve-self-signed-certs\""
Nov 03 20:52:00.801546 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.801516    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b64da81f-0a97-4cdd-9afd-1d58a306c717-tls-certs\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.801610 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.801597    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-model-cache\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.801750 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.801725    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5mdc\" (UniqueName: \"kubernetes.io/projected/b64da81f-0a97-4cdd-9afd-1d58a306c717-kube-api-access-j5mdc\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.801809 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.801792    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-dshm\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.801867 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.801851    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-tmp-dir\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.801923 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.801914    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-home\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.803331 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.803306    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"]
Nov 03 20:52:00.903173 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.903142    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-home\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.903355 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.903220    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b64da81f-0a97-4cdd-9afd-1d58a306c717-tls-certs\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.903355 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.903306    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-model-cache\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.903355 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.903326    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5mdc\" (UniqueName: \"kubernetes.io/projected/b64da81f-0a97-4cdd-9afd-1d58a306c717-kube-api-access-j5mdc\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.903533 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.903359    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-dshm\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.903533 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.903381    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-tmp-dir\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.903658 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.903598    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-home\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.903777 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.903747    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-model-cache\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.903855 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.903778    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-tmp-dir\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.906079 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.906052    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-dshm\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.906315 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.906294    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b64da81f-0a97-4cdd-9afd-1d58a306c717-tls-certs\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:00.964329 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:00.964300    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5mdc\" (UniqueName: \"kubernetes.io/projected/b64da81f-0a97-4cdd-9afd-1d58a306c717-kube-api-access-j5mdc\") pod \"simulator-w6-12a423-kserve-7b547b585d-2t5kd\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") " pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:01.019465 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:01.019433    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:01.406079 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:01.406052    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"]
Nov 03 20:52:01.408049 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:52:01.408019    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb64da81f_0a97_4cdd_9afd_1d58a306c717.slice/crio-067c38117a8148cae649bdba60fbecd92ef6375474bfe486be2a2ae7a828affa WatchSource:0}: Error finding container 067c38117a8148cae649bdba60fbecd92ef6375474bfe486be2a2ae7a828affa: Status 404 returned error can't find the container with id 067c38117a8148cae649bdba60fbecd92ef6375474bfe486be2a2ae7a828affa
Nov 03 20:52:01.669672 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:01.669561    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" event={"ID":"b64da81f-0a97-4cdd-9afd-1d58a306c717","Type":"ContainerStarted","Data":"067c38117a8148cae649bdba60fbecd92ef6375474bfe486be2a2ae7a828affa"}
Nov 03 20:52:01.670999 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:52:01.670967    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:52:04.687488 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:04.687410    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" event={"ID":"b64da81f-0a97-4cdd-9afd-1d58a306c717","Type":"ContainerStarted","Data":"c0a7bd9a7fccc6ef9659fef3a769eec695eb197c4acf11f42c589c323cfc1fa5"}
Nov 03 20:52:04.807562 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:04.807494    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podStartSLOduration=2.583872642 podStartE2EDuration="4.807465463s" totalImagesPullingTime="2.223592821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:52:04.80687087 +0000 UTC m=+727.351577263" watchObservedRunningTime="2026-11-03 20:52:04.807465463 +0000 UTC m=+727.352171834"
Nov 03 20:52:05.019555 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:05.019524    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:11.020219 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:11.020182    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:11.038720 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:11.038692    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:11.730815 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:11.730784    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 20:52:30.571916 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.571885    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"]
Nov 03 20:52:30.577094 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.577071    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.631706 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.631672    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-89bb58\"/\"openshift-service-ca.crt\""
Nov 03 20:52:30.681399 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.681366    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-89bb58\"/\"default-dockercfg-n6bg4\""
Nov 03 20:52:30.681399 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.681369    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-89bb58\"/\"simulator-w3-89bb58-kserve-self-signed-certs\""
Nov 03 20:52:30.681596 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.681516    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-89bb58\"/\"kube-root-ca.crt\""
Nov 03 20:52:30.683103 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.683079    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"]
Nov 03 20:52:30.684936 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.684918    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-model-cache\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.685016 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.684949    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-tls-certs\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.685100 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.685081    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49bgb\" (UniqueName: \"kubernetes.io/projected/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-kube-api-access-49bgb\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.685135 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.685116    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-dshm\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.685198 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.685180    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-tmp-dir\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.685252 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.685228    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-home\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.786214 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.786181    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49bgb\" (UniqueName: \"kubernetes.io/projected/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-kube-api-access-49bgb\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.786214 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.786222    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-dshm\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.786469 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.786259    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-tmp-dir\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.786469 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.786304    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-home\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.786469 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.786362    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-model-cache\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.786469 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.786386    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-tls-certs\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.786751 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.786727    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-home\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.786813 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.786770    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-tmp-dir\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.786871 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.786824    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-model-cache\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.788821 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.788798    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-dshm\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.789180 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.789160    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-tls-certs\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.847361 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.847292    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49bgb\" (UniqueName: \"kubernetes.io/projected/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-kube-api-access-49bgb\") pod \"simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") " pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:30.889364 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:30.889290    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:31.275917 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:31.275892    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"]
Nov 03 20:52:31.279025 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:52:31.278997    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38eea1db_b2e8_4364_a7a9_b7bb655a20f1.slice/crio-622e19c0cc850813b2622703a027e091c3c90b61d2212c4551069e49b583da27 WatchSource:0}: Error finding container 622e19c0cc850813b2622703a027e091c3c90b61d2212c4551069e49b583da27: Status 404 returned error can't find the container with id 622e19c0cc850813b2622703a027e091c3c90b61d2212c4551069e49b583da27
Nov 03 20:52:31.669899 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:52:31.669807    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:52:31.825192 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:31.825162    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" event={"ID":"38eea1db-b2e8-4364-a7a9-b7bb655a20f1","Type":"ContainerStarted","Data":"c8f1f1b552efd36d52e1a51a53bb21d7267eb7edd1d916b6c1ef27aa088d4575"}
Nov 03 20:52:31.825373 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:31.825202    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" event={"ID":"38eea1db-b2e8-4364-a7a9-b7bb655a20f1","Type":"ContainerStarted","Data":"622e19c0cc850813b2622703a027e091c3c90b61d2212c4551069e49b583da27"}
Nov 03 20:52:31.889777 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:31.889743    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:31.944577 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:31.944483    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podStartSLOduration=1.944470127 podStartE2EDuration="1.944470127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:52:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:52:31.941824888 +0000 UTC m=+754.486531257" watchObservedRunningTime="2026-11-03 20:52:31.944470127 +0000 UTC m=+754.489176498"
Nov 03 20:52:33.969686 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:33.969648    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"]
Nov 03 20:52:33.973461 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:33.973440    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.013168 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.013139    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-model-cache\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.013332 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.013199    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-dshm\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.013332 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.013310    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0c71be3-4788-4e30-8bfb-905e950a6d0a-tls-certs\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.013429 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.013362    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rxl2\" (UniqueName: \"kubernetes.io/projected/c0c71be3-4788-4e30-8bfb-905e950a6d0a-kube-api-access-7rxl2\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.013468 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.013425    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-home\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.013468 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.013453    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-tmp-dir\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.071481 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.071450    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-d01cef\"/\"openshift-service-ca.crt\""
Nov 03 20:52:34.075952 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.075932    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-d01cef\"/\"kube-root-ca.crt\""
Nov 03 20:52:34.076075 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.076042    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-d01cef\"/\"default-dockercfg-2qc48\""
Nov 03 20:52:34.076406 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.076387    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-d01cef\"/\"simulator-w1-d01cef-kserve-self-signed-certs\""
Nov 03 20:52:34.078674 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.078652    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"]
Nov 03 20:52:34.114372 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.114335    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0c71be3-4788-4e30-8bfb-905e950a6d0a-tls-certs\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.114660 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.114404    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rxl2\" (UniqueName: \"kubernetes.io/projected/c0c71be3-4788-4e30-8bfb-905e950a6d0a-kube-api-access-7rxl2\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.114660 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.114435    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-home\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.114660 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.114465    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-tmp-dir\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.114660 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.114546    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-model-cache\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.114660 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.114616    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-dshm\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.115158 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.115132    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-tmp-dir\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.115362 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.115338    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-home\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.115438 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.115359    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-model-cache\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.117592 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.117563    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-dshm\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.118079 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.118059    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0c71be3-4788-4e30-8bfb-905e950a6d0a-tls-certs\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.177176 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.177143    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rxl2\" (UniqueName: \"kubernetes.io/projected/c0c71be3-4788-4e30-8bfb-905e950a6d0a-kube-api-access-7rxl2\") pod \"simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") " pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.285991 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.285894    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:34.679549 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.679525    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"]
Nov 03 20:52:34.682209 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:52:34.682179    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0c71be3_4788_4e30_8bfb_905e950a6d0a.slice/crio-0fb2f7fe219ee9ffbbe619f1480d72936453f5897f0f2390bafe72000fe36a5b WatchSource:0}: Error finding container 0fb2f7fe219ee9ffbbe619f1480d72936453f5897f0f2390bafe72000fe36a5b: Status 404 returned error can't find the container with id 0fb2f7fe219ee9ffbbe619f1480d72936453f5897f0f2390bafe72000fe36a5b
Nov 03 20:52:34.840823 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.840786    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" event={"ID":"c0c71be3-4788-4e30-8bfb-905e950a6d0a","Type":"ContainerStarted","Data":"a53e586a99392a3d95ee9c5c59b8043638992da9fdf8289716db416934c7059f"}
Nov 03 20:52:34.840823 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.840826    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" event={"ID":"c0c71be3-4788-4e30-8bfb-905e950a6d0a","Type":"ContainerStarted","Data":"0fb2f7fe219ee9ffbbe619f1480d72936453f5897f0f2390bafe72000fe36a5b"}
Nov 03 20:52:34.959460 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:34.959330    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podStartSLOduration=1.95931369 podStartE2EDuration="1.95931369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:52:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:52:34.958352745 +0000 UTC m=+757.503059115" watchObservedRunningTime="2026-11-03 20:52:34.95931369 +0000 UTC m=+757.504020061"
Nov 03 20:52:35.286614 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:35.286585    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:40.890398 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:40.890356    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:40.903196 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:40.903174    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:41.888858 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:41.888829    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 20:52:44.286286 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:44.286248    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:44.299141 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:44.299116    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:44.902363 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:44.902303    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 20:52:55.622492 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.622460    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"]
Nov 03 20:52:55.626387 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.626370    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.712905 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.712867    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2204dd40-a737-47fc-a685-420d5b819a29-tls-certs\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.713054 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.712925    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-tmp-dir\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.713054 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.713032    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-home\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.713164 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.713085    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-dshm\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.713298 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.713275    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-model-cache\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.713368 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.713327    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx6dj\" (UniqueName: \"kubernetes.io/projected/2204dd40-a737-47fc-a685-420d5b819a29-kube-api-access-dx6dj\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.729030 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.728998    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-e22abf\"/\"default-dockercfg-gdxxs\""
Nov 03 20:52:55.729030 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.729008    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-e22abf\"/\"simulator-w5-e22abf-kserve-self-signed-certs\""
Nov 03 20:52:55.729233 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.729051    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-e22abf\"/\"openshift-service-ca.crt\""
Nov 03 20:52:55.729233 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.729118    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-e22abf\"/\"kube-root-ca.crt\""
Nov 03 20:52:55.731052 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.731030    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"]
Nov 03 20:52:55.814948 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.814904    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-model-cache\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.815147 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.814970    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx6dj\" (UniqueName: \"kubernetes.io/projected/2204dd40-a737-47fc-a685-420d5b819a29-kube-api-access-dx6dj\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.815147 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.815004    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2204dd40-a737-47fc-a685-420d5b819a29-tls-certs\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.815147 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.815042    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-tmp-dir\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.815147 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.815082    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-home\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.815147 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.815129    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-dshm\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.816095 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.816068    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-model-cache\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.816699 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.816671    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-home\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.816905 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.816885    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-tmp-dir\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.818875 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.818850    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2204dd40-a737-47fc-a685-420d5b819a29-tls-certs\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.819972 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.819944    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-dshm\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.878043 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.877962    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx6dj\" (UniqueName: \"kubernetes.io/projected/2204dd40-a737-47fc-a685-420d5b819a29-kube-api-access-dx6dj\") pod \"simulator-w5-e22abf-kserve-67d48d4c4-jrpgv\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") " pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:55.938791 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:55.938761    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:52:56.333656 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:56.333176    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"]
Nov 03 20:52:56.336325 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:52:56.336301    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2204dd40_a737_47fc_a685_420d5b819a29.slice/crio-96e1f18265ad9a47fc3913953bda5797b19ef016aa9a8404ae62c8c05ddd419c WatchSource:0}: Error finding container 96e1f18265ad9a47fc3913953bda5797b19ef016aa9a8404ae62c8c05ddd419c: Status 404 returned error can't find the container with id 96e1f18265ad9a47fc3913953bda5797b19ef016aa9a8404ae62c8c05ddd419c
Nov 03 20:52:56.951938 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:56.951903    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" event={"ID":"2204dd40-a737-47fc-a685-420d5b819a29","Type":"ContainerStarted","Data":"0ef61ee1499c0063b5a8521797cc65fff45e42f1d85e633f1130cc7a0edf392a"}
Nov 03 20:52:56.952378 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:56.951951    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" event={"ID":"2204dd40-a737-47fc-a685-420d5b819a29","Type":"ContainerStarted","Data":"96e1f18265ad9a47fc3913953bda5797b19ef016aa9a8404ae62c8c05ddd419c"}
Nov 03 20:52:57.085242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:57.085195    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podStartSLOduration=2.085181039 podStartE2EDuration="2.085181039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:52:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:52:57.083217876 +0000 UTC m=+779.627924247" watchObservedRunningTime="2026-11-03 20:52:57.085181039 +0000 UTC m=+779.629887426"
Nov 03 20:52:57.939132 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:52:57.939098    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:53:01.670282 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:53:01.670248    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:53:05.939311 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:05.939277    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:53:05.951766 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:05.951743    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:53:06.012307 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:06.012274    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 20:53:09.153093 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.153058    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"]
Nov 03 20:53:09.156314 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.156298    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.242775 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.242736    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-dshm\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.242940 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.242908    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-home\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.242940 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.242930    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/809df5f7-98f7-4dbf-8548-b0f7dade9c40-tls-certs\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.243027 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.242948    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-model-cache\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.243027 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.242969    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-tmp-dir\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.243027 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.242991    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgqbb\" (UniqueName: \"kubernetes.io/projected/809df5f7-98f7-4dbf-8548-b0f7dade9c40-kube-api-access-wgqbb\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.258536 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.258509    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-12a423\"/\"premium-w6-12a423-kserve-self-signed-certs\""
Nov 03 20:53:09.260347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.260327    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"]
Nov 03 20:53:09.343976 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.343943    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-home\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.343976 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.343976    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/809df5f7-98f7-4dbf-8548-b0f7dade9c40-tls-certs\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.344231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.343994    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-model-cache\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.344231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.344013    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-tmp-dir\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.344231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.344035    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgqbb\" (UniqueName: \"kubernetes.io/projected/809df5f7-98f7-4dbf-8548-b0f7dade9c40-kube-api-access-wgqbb\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.344231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.344084    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-dshm\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.344458 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.344410    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-home\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.344458 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.344440    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-model-cache\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.344552 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.344472    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-tmp-dir\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.346513 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.346486    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-dshm\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.346674 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.346580    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/809df5f7-98f7-4dbf-8548-b0f7dade9c40-tls-certs\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.405196 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.405133    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgqbb\" (UniqueName: \"kubernetes.io/projected/809df5f7-98f7-4dbf-8548-b0f7dade9c40-kube-api-access-wgqbb\") pod \"premium-w6-12a423-kserve-869cd6648c-hj2hk\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") " pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.468270 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.468237    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:09.873096 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:09.873073    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"]
Nov 03 20:53:09.875646 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:53:09.875605    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod809df5f7_98f7_4dbf_8548_b0f7dade9c40.slice/crio-c958689a22672fe577223e4f622d5fcf0c5615236036676b8e154b4127d221c9 WatchSource:0}: Error finding container c958689a22672fe577223e4f622d5fcf0c5615236036676b8e154b4127d221c9: Status 404 returned error can't find the container with id c958689a22672fe577223e4f622d5fcf0c5615236036676b8e154b4127d221c9
Nov 03 20:53:10.021498 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:10.021459    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" event={"ID":"809df5f7-98f7-4dbf-8548-b0f7dade9c40","Type":"ContainerStarted","Data":"d392c6f8a4fe5ffa8d4cf7acb16aba418f3bd7cf826961181f8ead5408fd5915"}
Nov 03 20:53:10.021498 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:10.021505    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" event={"ID":"809df5f7-98f7-4dbf-8548-b0f7dade9c40","Type":"ContainerStarted","Data":"c958689a22672fe577223e4f622d5fcf0c5615236036676b8e154b4127d221c9"}
Nov 03 20:53:10.139683 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:10.139530    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podStartSLOduration=1.139515056 podStartE2EDuration="1.139515056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:53:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:53:10.137695096 +0000 UTC m=+792.682401466" watchObservedRunningTime="2026-11-03 20:53:10.139515056 +0000 UTC m=+792.684221451"
Nov 03 20:53:10.469209 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:10.469176    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:19.468898 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:19.468853    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:19.481541 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:19.481520    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:20.084325 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:53:20.084293    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 20:53:31.670224 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:53:31.670186    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:54:01.670282 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:54:01.670243    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:54:01.949528 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:01.949449    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"]
Nov 03 20:54:01.953463 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:01.953441    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.038191 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.038157    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v84hq\" (UniqueName: \"kubernetes.io/projected/df7bec60-2e19-4184-9432-a08efd6ea203-kube-api-access-v84hq\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.038374 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.038203    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-tmp-dir\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.038374 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.038349    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-model-cache\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.038478 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.038397    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-dshm\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.038519 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.038491    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-home\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.038554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.038539    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df7bec60-2e19-4184-9432-a08efd6ea203-tls-certs\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.055188 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.055162    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-499187\"/\"default-dockercfg-6gx7b\""
Nov 03 20:54:02.055358 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.055320    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-499187\"/\"premium-w0-499187-kserve-self-signed-certs\""
Nov 03 20:54:02.055427 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.055405    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-499187\"/\"openshift-service-ca.crt\""
Nov 03 20:54:02.055472 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.055429    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-499187\"/\"kube-root-ca.crt\""
Nov 03 20:54:02.057710 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.057690    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"]
Nov 03 20:54:02.139776 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.139745    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v84hq\" (UniqueName: \"kubernetes.io/projected/df7bec60-2e19-4184-9432-a08efd6ea203-kube-api-access-v84hq\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.139975 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.139782    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-tmp-dir\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.139975 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.139835    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-model-cache\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.139975 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.139856    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-dshm\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.139975 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.139872    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-home\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.140186 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.140019    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df7bec60-2e19-4184-9432-a08efd6ea203-tls-certs\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.140308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.140285    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-home\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.140434 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.140338    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-tmp-dir\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.140434 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.140356    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-model-cache\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.142254 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.142225    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-dshm\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.142486 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.142470    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df7bec60-2e19-4184-9432-a08efd6ea203-tls-certs\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.201201 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.201126    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v84hq\" (UniqueName: \"kubernetes.io/projected/df7bec60-2e19-4184-9432-a08efd6ea203-kube-api-access-v84hq\") pod \"premium-w0-499187-kserve-7774c96bc4-5k27x\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") " pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.265685 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.265654    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:02.659158 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:02.659133    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"]
Nov 03 20:54:02.661803 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:54:02.661765    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf7bec60_2e19_4184_9432_a08efd6ea203.slice/crio-8aa81e4a223bee7d47e889297a3c75845b1af9a9ca736bb50702efb557d6326d WatchSource:0}: Error finding container 8aa81e4a223bee7d47e889297a3c75845b1af9a9ca736bb50702efb557d6326d: Status 404 returned error can't find the container with id 8aa81e4a223bee7d47e889297a3c75845b1af9a9ca736bb50702efb557d6326d
Nov 03 20:54:03.308397 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:03.308356    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" event={"ID":"df7bec60-2e19-4184-9432-a08efd6ea203","Type":"ContainerStarted","Data":"56273e48a1c233067e26ba0d5a570a6f67b14a6cd52f8987b1936de2e7bdff68"}
Nov 03 20:54:03.308924 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:03.308409    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" event={"ID":"df7bec60-2e19-4184-9432-a08efd6ea203","Type":"ContainerStarted","Data":"8aa81e4a223bee7d47e889297a3c75845b1af9a9ca736bb50702efb557d6326d"}
Nov 03 20:54:03.428205 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:03.428153    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podStartSLOduration=2.428134179 podStartE2EDuration="2.428134179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:54:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:54:03.426869004 +0000 UTC m=+845.971575376" watchObservedRunningTime="2026-11-03 20:54:03.428134179 +0000 UTC m=+845.972840550"
Nov 03 20:54:04.266896 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:04.266844    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:12.266461 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:12.266425    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:12.279027 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:12.279004    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:12.364435 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:12.364408    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 20:54:31.669965 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:54:31.669927    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:54:34.780562 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.780524    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"]
Nov 03 20:54:34.784658 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.784640    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.881747 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.881708    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-model-cache\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.881972 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.881790    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-home\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.881972 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.881888    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-tmp-dir\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.881972 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.881957    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-dshm\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.882161 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.882022    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdzwt\" (UniqueName: \"kubernetes.io/projected/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-kube-api-access-cdzwt\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.882161 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.882144    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-tls-certs\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.889605 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.889573    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-ca4642\"/\"premium-w4-ca4642-kserve-self-signed-certs\""
Nov 03 20:54:34.889605 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.889573    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-ca4642\"/\"default-dockercfg-nq5lb\""
Nov 03 20:54:34.890195 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.890153    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-ca4642\"/\"kube-root-ca.crt\""
Nov 03 20:54:34.890279 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.890265    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-ca4642\"/\"openshift-service-ca.crt\""
Nov 03 20:54:34.892464 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.892445    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"]
Nov 03 20:54:34.982874 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.982832    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-home\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.983072 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.982885    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-tmp-dir\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.983072 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.983014    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-dshm\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.983219 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.983096    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdzwt\" (UniqueName: \"kubernetes.io/projected/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-kube-api-access-cdzwt\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.983219 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.983190    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-tls-certs\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.983314 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.983221    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-model-cache\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.983352 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.983318    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-home\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.983395 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.983302    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-tmp-dir\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.983585 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.983566    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-model-cache\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.985619 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.985590    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-dshm\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:34.986173 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:34.986152    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-tls-certs\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:35.048927 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:35.048855    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdzwt\" (UniqueName: \"kubernetes.io/projected/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-kube-api-access-cdzwt\") pod \"premium-w4-ca4642-kserve-57b744747d-s9969\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") " pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:35.097431 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:35.097393    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:35.505049 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:35.505022    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"]
Nov 03 20:54:35.508979 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:54:35.508953    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod933b980b_58f5_44d0_b7b6_9f3fd58bcb63.slice/crio-9adf7ce9a3bf80d4a2b76c4797a24463a47de8923e61287e4c459b1c7cfc3943 WatchSource:0}: Error finding container 9adf7ce9a3bf80d4a2b76c4797a24463a47de8923e61287e4c459b1c7cfc3943: Status 404 returned error can't find the container with id 9adf7ce9a3bf80d4a2b76c4797a24463a47de8923e61287e4c459b1c7cfc3943
Nov 03 20:54:36.490380 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:36.490337    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" event={"ID":"933b980b-58f5-44d0-b7b6-9f3fd58bcb63","Type":"ContainerStarted","Data":"e5672705e69f44aa10e25023c7ef4a69c286ec738d49d99bd6b0b441d00fb30f"}
Nov 03 20:54:36.490380 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:36.490384    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" event={"ID":"933b980b-58f5-44d0-b7b6-9f3fd58bcb63","Type":"ContainerStarted","Data":"9adf7ce9a3bf80d4a2b76c4797a24463a47de8923e61287e4c459b1c7cfc3943"}
Nov 03 20:54:36.609019 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:36.608961    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podStartSLOduration=2.608947056 podStartE2EDuration="2.608947056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:54:36.607396768 +0000 UTC m=+879.152103139" watchObservedRunningTime="2026-11-03 20:54:36.608947056 +0000 UTC m=+879.153653425"
Nov 03 20:54:37.097729 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:37.097701    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:45.098542 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:45.098508    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:45.110703 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:45.110681    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:45.550559 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:45.550529    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 20:54:52.909412 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:52.909379    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"]
Nov 03 20:54:52.913002 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:52.912986    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:52.966033 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:52.966003    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-862138\"/\"kube-root-ca.crt\""
Nov 03 20:54:52.995643 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:52.995593    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-862138\"/\"openshift-service-ca.crt\""
Nov 03 20:54:53.015741 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.015693    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-862138\"/\"premium-w2-862138-kserve-self-signed-certs\""
Nov 03 20:54:53.015741 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.015728    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-862138\"/\"default-dockercfg-fsbwz\""
Nov 03 20:54:53.017701 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.017685    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"]
Nov 03 20:54:53.077390 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.077357    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff97541-97f9-4ff6-b218-c717565ef6a4-tls-certs\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.077554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.077489    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-tmp-dir\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.077554 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.077514    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-home\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.077654 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.077637    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-model-cache\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.077696 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.077676    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ktbs\" (UniqueName: \"kubernetes.io/projected/4ff97541-97f9-4ff6-b218-c717565ef6a4-kube-api-access-8ktbs\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.077757 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.077742    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-dshm\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.178619 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.178539    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-model-cache\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.178619 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.178577    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ktbs\" (UniqueName: \"kubernetes.io/projected/4ff97541-97f9-4ff6-b218-c717565ef6a4-kube-api-access-8ktbs\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.178619 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.178609    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-dshm\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.178932 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.178660    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff97541-97f9-4ff6-b218-c717565ef6a4-tls-certs\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.178932 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.178723    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-tmp-dir\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.178932 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.178746    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-home\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.179091 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.179068    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-model-cache\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.179147 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.179101    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-home\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.179147 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.179133    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-tmp-dir\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.181164 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.181129    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-dshm\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.181323 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.181307    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff97541-97f9-4ff6-b218-c717565ef6a4-tls-certs\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.240563 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.240528    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ktbs\" (UniqueName: \"kubernetes.io/projected/4ff97541-97f9-4ff6-b218-c717565ef6a4-kube-api-access-8ktbs\") pod \"premium-w2-862138-kserve-5dd7d4cdd-scr2n\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") " pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.526231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.526198    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:54:53.928705 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:53.928605    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"]
Nov 03 20:54:53.931600 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:54:53.931570    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ff97541_97f9_4ff6_b218_c717565ef6a4.slice/crio-2af784417810a3be5bf4c4e9c8686a74e60f4960495cfc7cb4fb65596d5695b2 WatchSource:0}: Error finding container 2af784417810a3be5bf4c4e9c8686a74e60f4960495cfc7cb4fb65596d5695b2: Status 404 returned error can't find the container with id 2af784417810a3be5bf4c4e9c8686a74e60f4960495cfc7cb4fb65596d5695b2
Nov 03 20:54:54.592225 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:54.592185    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" event={"ID":"4ff97541-97f9-4ff6-b218-c717565ef6a4","Type":"ContainerStarted","Data":"037cd997fbc22fe0fdf8443cecc5a602c4dd3b09095e1abc626464ebcdc68801"}
Nov 03 20:54:54.592412 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:54.592239    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" event={"ID":"4ff97541-97f9-4ff6-b218-c717565ef6a4","Type":"ContainerStarted","Data":"2af784417810a3be5bf4c4e9c8686a74e60f4960495cfc7cb4fb65596d5695b2"}
Nov 03 20:54:54.716189 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:54.716137    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podStartSLOduration=2.716124856 podStartE2EDuration="2.716124856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:54:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:54:54.715575086 +0000 UTC m=+897.260281457" watchObservedRunningTime="2026-11-03 20:54:54.716124856 +0000 UTC m=+897.260831225"
Nov 03 20:54:55.527087 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:54:55.527057    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:55:01.672161 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:55:01.672119    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:55:03.527392 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:55:03.527352    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:55:03.539716 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:55:03.539687    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:55:03.653223 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:55:03.653191    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 20:55:31.670781 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:55:31.670745    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:56:01.672585 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:56:01.672550    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:56:03.537700 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.537665    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"]
Nov 03 20:56:03.541493 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.541472    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.645798 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.645758    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"]
Nov 03 20:56:03.656681 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.656656    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-12a423-openshift-default-dockercfg-2ztvn\""
Nov 03 20:56:03.673231 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.673202    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st5gt\" (UniqueName: \"kubernetes.io/projected/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-kube-api-access-st5gt\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.673370 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.673249    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.673464 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.673386    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.673502 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.673471    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.673535 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.673505    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.673535 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.673529    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.673609 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.673545    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.673726 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.673706    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.673784 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.673755    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.775013 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.774983    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.775195 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.775031    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.775195 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.775052    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.775195 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.775074    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.775195 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.775090    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.775195 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.775131    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.775195 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.775166    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.775501 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.775219    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st5gt\" (UniqueName: \"kubernetes.io/projected/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-kube-api-access-st5gt\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.775501 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.775259    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.775501 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.775452    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.776147 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.776120    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.776275 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.776244    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.776401 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.776156    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.776401 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.776147    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.777987 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.777962    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.778587 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.778565    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.836492 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.836410    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.836646 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.836547    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-st5gt\" (UniqueName: \"kubernetes.io/projected/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-kube-api-access-st5gt\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:03.854368 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:03.854341    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:04.291177 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:04.291141    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"]
Nov 03 20:56:04.292883 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:56:04.292852    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b3af9c_bf26_4963_9146_7dadd2eb20ba.slice/crio-da4e23b111682eb8eaf9c64e96ddeadbc160edc4513a4f2a364afda7982deebe WatchSource:0}: Error finding container da4e23b111682eb8eaf9c64e96ddeadbc160edc4513a4f2a364afda7982deebe: Status 404 returned error can't find the container with id da4e23b111682eb8eaf9c64e96ddeadbc160edc4513a4f2a364afda7982deebe
Nov 03 20:56:04.295357 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:04.295324    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:04.295450 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:04.295408    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:04.295519 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:04.295457    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:04.988233 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:04.988191    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g" event={"ID":"f9b3af9c-bf26-4963-9146-7dadd2eb20ba","Type":"ContainerStarted","Data":"39c6b389d43c8145088ed71b2454988e3180da2a7ec25af24b83b89e30d2481e"}
Nov 03 20:56:04.989255 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:04.989229    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g" event={"ID":"f9b3af9c-bf26-4963-9146-7dadd2eb20ba","Type":"ContainerStarted","Data":"da4e23b111682eb8eaf9c64e96ddeadbc160edc4513a4f2a364afda7982deebe"}
Nov 03 20:56:05.169289 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:05.169224    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g" podStartSLOduration=2.169204468 podStartE2EDuration="2.169204468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:56:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:56:05.167103949 +0000 UTC m=+967.711810319" watchObservedRunningTime="2026-11-03 20:56:05.169204468 +0000 UTC m=+967.713910836"
Nov 03 20:56:05.854689 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:05.854649    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:05.856471 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:05.856446    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 03 20:56:05.856581 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:05.856499    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g" podUID="f9b3af9c-bf26-4963-9146-7dadd2eb20ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 03 20:56:06.855034 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:06.854987    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 03 20:56:06.855692 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:06.855061    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g" podUID="f9b3af9c-bf26-4963-9146-7dadd2eb20ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 03 20:56:07.855254 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:07.855208    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 03 20:56:07.855817 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:07.855270    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g" podUID="f9b3af9c-bf26-4963-9146-7dadd2eb20ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 03 20:56:08.855269 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:08.855229    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 03 20:56:08.855738 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:08.855300    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g" podUID="f9b3af9c-bf26-4963-9146-7dadd2eb20ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 03 20:56:09.855536 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:09.855486    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 03 20:56:09.856033 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:09.855580    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g" podUID="f9b3af9c-bf26-4963-9146-7dadd2eb20ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 03 20:56:10.855497 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:10.855465    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Nov 03 20:56:10.855719 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:10.855558    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g" podUID="f9b3af9c-bf26-4963-9146-7dadd2eb20ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Nov 03 20:56:11.859531 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:11.859491    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:11.859973 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:11.859872    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:11.860422 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:11.860401    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 20:56:19.577221 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.577193    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"]
Nov 03 20:56:19.581244 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.581220    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.590413 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.590390    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.590489 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.590433    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n92d4\" (UniqueName: \"kubernetes.io/projected/98f5c9db-d68a-4f60-8bde-d8a25acd1275-kube-api-access-n92d4\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.590602 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.590583    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.590721 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.590700    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.590759 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.590734    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.590869 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.590848    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.590906 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.590893    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.590988 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.590909    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.590988 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.590951    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.690989 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.690956    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"]
Nov 03 20:56:19.696660 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.696602    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.696805 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.696719    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.696805 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.696749    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.696938 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.696843    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.696938 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.696895    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.696938 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.696922    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.697101 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.696946    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.697101 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.697023    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.697101 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.697047    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n92d4\" (UniqueName: \"kubernetes.io/projected/98f5c9db-d68a-4f60-8bde-d8a25acd1275-kube-api-access-n92d4\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.697837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.697813    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.698481 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.698458    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.698741 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.698714    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.699009 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.698981    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.699177 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.699151    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.704178 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.704147    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.704498 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.704468    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.759005 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.758976    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.759312 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.759285    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n92d4\" (UniqueName: \"kubernetes.io/projected/98f5c9db-d68a-4f60-8bde-d8a25acd1275-kube-api-access-n92d4\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:19.897813 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:19.897704    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:20.329912 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:20.329879    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"]
Nov 03 20:56:20.335015 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:56:20.334982    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f5c9db_d68a_4f60_8bde_d8a25acd1275.slice/crio-274f64d2a8d99f0552cf447b88a4e63c69fda716e182c38e07fcc109ac09b1ea WatchSource:0}: Error finding container 274f64d2a8d99f0552cf447b88a4e63c69fda716e182c38e07fcc109ac09b1ea: Status 404 returned error can't find the container with id 274f64d2a8d99f0552cf447b88a4e63c69fda716e182c38e07fcc109ac09b1ea
Nov 03 20:56:20.337584 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:20.337545    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:20.337701 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:20.337663    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:20.337765 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:20.337717    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:21.112317 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:21.112263    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x" event={"ID":"98f5c9db-d68a-4f60-8bde-d8a25acd1275","Type":"ContainerStarted","Data":"6cbe7a39468bf3dfc11e430b71641613d359f5b70cc4d72e73e3efa2fe2e3f75"}
Nov 03 20:56:21.112770 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:21.112354    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x" event={"ID":"98f5c9db-d68a-4f60-8bde-d8a25acd1275","Type":"ContainerStarted","Data":"274f64d2a8d99f0552cf447b88a4e63c69fda716e182c38e07fcc109ac09b1ea"}
Nov 03 20:56:21.300667 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:21.298272    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x" podStartSLOduration=2.298237527 podStartE2EDuration="2.298237527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:56:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:56:21.23593443 +0000 UTC m=+983.780640803" watchObservedRunningTime="2026-11-03 20:56:21.298237527 +0000 UTC m=+983.842943896"
Nov 03 20:56:21.899973 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:21.899933    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:21.901915 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:21.901871    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 03 20:56:21.902068 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:21.901934    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x" podUID="98f5c9db-d68a-4f60-8bde-d8a25acd1275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 03 20:56:22.898823 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:22.898786    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 03 20:56:22.899312 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:22.898855    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x" podUID="98f5c9db-d68a-4f60-8bde-d8a25acd1275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 03 20:56:23.899123 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:23.898901    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 03 20:56:23.899123 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:23.898957    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x" podUID="98f5c9db-d68a-4f60-8bde-d8a25acd1275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 03 20:56:24.898576 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:24.898531    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 03 20:56:24.898799 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:24.898597    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x" podUID="98f5c9db-d68a-4f60-8bde-d8a25acd1275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 03 20:56:25.904030 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:25.903996    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 03 20:56:25.904607 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:25.904571    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x" podUID="98f5c9db-d68a-4f60-8bde-d8a25acd1275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 03 20:56:26.898957 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:26.898922    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 03 20:56:26.899152 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:26.898970    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x" podUID="98f5c9db-d68a-4f60-8bde-d8a25acd1275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 03 20:56:27.898967 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:27.898928    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Nov 03 20:56:27.899367 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:27.898989    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x" podUID="98f5c9db-d68a-4f60-8bde-d8a25acd1275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Nov 03 20:56:29.104967 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:29.104921    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:29.105490 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:29.105471    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:29.106514 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:29.106489    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 20:56:31.671371 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:56:31.671320    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:56:35.499388 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.499309    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"]
Nov 03 20:56:35.504148 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.504130    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.556470 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.556445    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-d01cef-openshift-default-dockercfg-bxtfs\""
Nov 03 20:56:35.604918 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.604879    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.605060 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.604971    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.605060 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.605020    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr9nc\" (UniqueName: \"kubernetes.io/projected/4387fbe5-ef9e-4b48-9251-8181f1f941f9-kube-api-access-jr9nc\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.605060 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.605054    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.605234 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.605081    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.605234 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.605176    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.605234 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.605206    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.605399 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.605240    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.605399 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.605281    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.609356 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.609327    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"]
Nov 03 20:56:35.706794 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.706750    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr9nc\" (UniqueName: \"kubernetes.io/projected/4387fbe5-ef9e-4b48-9251-8181f1f941f9-kube-api-access-jr9nc\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.706981 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.706807    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.706981 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.706841    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.706981 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.706957    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.707158 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.706987    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.707158 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.707020    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.707158 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.707049    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.707158 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.707090    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.707338 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.707175    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.707591 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.707563    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.707716 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.707588    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.707716 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.707662    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.708107 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.708081    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.708233 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.708175    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.709966 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.709937    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.711759 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.711733    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.767919 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.767844    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.768196 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.768173    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr9nc\" (UniqueName: \"kubernetes.io/projected/4387fbe5-ef9e-4b48-9251-8181f1f941f9-kube-api-access-jr9nc\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:35.818005 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:35.817971    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:36.249530 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:36.249501    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"]
Nov 03 20:56:36.251484 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:56:36.251454    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4387fbe5_ef9e_4b48_9251_8181f1f941f9.slice/crio-1a9e0c02f852807d7dfaa8c6d61cec5066a80c553ec6ab557b98000479211f48 WatchSource:0}: Error finding container 1a9e0c02f852807d7dfaa8c6d61cec5066a80c553ec6ab557b98000479211f48: Status 404 returned error can't find the container with id 1a9e0c02f852807d7dfaa8c6d61cec5066a80c553ec6ab557b98000479211f48
Nov 03 20:56:36.253663 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:36.253615    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:36.253718 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:36.253698    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:36.253757 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:36.253730    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:37.213863 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:37.213834    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq" event={"ID":"4387fbe5-ef9e-4b48-9251-8181f1f941f9","Type":"ContainerStarted","Data":"c8a9fcb96c9a9a0025185b95b04aa95c64141c96781bc8f24f867041551cba00"}
Nov 03 20:56:37.214264 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:37.213888    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq" event={"ID":"4387fbe5-ef9e-4b48-9251-8181f1f941f9","Type":"ContainerStarted","Data":"1a9e0c02f852807d7dfaa8c6d61cec5066a80c553ec6ab557b98000479211f48"}
Nov 03 20:56:37.818337 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:37.818303    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:37.819865 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:37.819843    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 03 20:56:37.819931 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:37.819891    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq" podUID="4387fbe5-ef9e-4b48-9251-8181f1f941f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 03 20:56:38.818810 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:38.818769    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 03 20:56:38.819438 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:38.818836    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq" podUID="4387fbe5-ef9e-4b48-9251-8181f1f941f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 03 20:56:39.819340 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:39.819308    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 03 20:56:39.819829 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:39.819791    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq" podUID="4387fbe5-ef9e-4b48-9251-8181f1f941f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 03 20:56:40.818274 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:40.818239    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 03 20:56:40.818440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:40.818290    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq" podUID="4387fbe5-ef9e-4b48-9251-8181f1f941f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 03 20:56:41.818696 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:41.818659    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Nov 03 20:56:41.819095 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:41.818715    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq" podUID="4387fbe5-ef9e-4b48-9251-8181f1f941f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Nov 03 20:56:42.968083 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:42.968023    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:42.969007 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:42.968979    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:42.970045 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:42.970021    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 20:56:43.141734 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:43.141391    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq" podStartSLOduration=8.141351479 podStartE2EDuration="8.141351479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:56:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:56:37.393954141 +0000 UTC m=+999.938660512" watchObservedRunningTime="2026-11-03 20:56:43.141351479 +0000 UTC m=+1005.686057850"
Nov 03 20:56:49.201418 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.201374    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"]
Nov 03 20:56:49.206019 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.205999    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.259837 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.259810    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-89bb58-openshift-default-dockercfg-26hlr\""
Nov 03 20:56:49.294565 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.294524    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.294747 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.294675    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.294747 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.294715    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.294884 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.294760    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.294884 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.294855    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.295014 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.294889    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.295014 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.294910    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l44s5\" (UniqueName: \"kubernetes.io/projected/d549e05f-a5a1-478c-bdb7-18b03af91b72-kube-api-access-l44s5\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.295014 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.294929    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.295169 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.295093    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d549e05f-a5a1-478c-bdb7-18b03af91b72-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.311703 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.311679    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"]
Nov 03 20:56:49.395975 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.395945    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.396117 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.395980    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.396117 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.396007    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.396117 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.396036    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.396117 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.396068    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.396117 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.396086    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l44s5\" (UniqueName: \"kubernetes.io/projected/d549e05f-a5a1-478c-bdb7-18b03af91b72-kube-api-access-l44s5\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.396117 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.396108    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.396403 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.396158    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d549e05f-a5a1-478c-bdb7-18b03af91b72-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.396403 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.396203    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.396511 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.396441    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.396511 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.396471    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.396785 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.396754    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.396906 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.396764    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.396906 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.396898    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d549e05f-a5a1-478c-bdb7-18b03af91b72-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.398621 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.398601    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.398778 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.398761    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.459224 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.459161    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l44s5\" (UniqueName: \"kubernetes.io/projected/d549e05f-a5a1-478c-bdb7-18b03af91b72-kube-api-access-l44s5\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.461465 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.461444    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.520229 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.520203    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:49.949482 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.949458    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"]
Nov 03 20:56:49.951410 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:56:49.951370    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd549e05f_a5a1_478c_bdb7_18b03af91b72.slice/crio-57eb621be490be8aaa546961d39bccb19921b34898c06f5e64a080aea4ea04f5 WatchSource:0}: Error finding container 57eb621be490be8aaa546961d39bccb19921b34898c06f5e64a080aea4ea04f5: Status 404 returned error can't find the container with id 57eb621be490be8aaa546961d39bccb19921b34898c06f5e64a080aea4ea04f5
Nov 03 20:56:49.953660 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.953606    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:49.953728 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.953694    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:49.953728 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:49.953724    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:56:50.293153 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:50.293115    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48" event={"ID":"d549e05f-a5a1-478c-bdb7-18b03af91b72","Type":"ContainerStarted","Data":"61f101a971bbe6978267554a0f431161bb6d8c00c6d16cb9e0583a08f1cb40d6"}
Nov 03 20:56:50.293560 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:50.293188    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48" event={"ID":"d549e05f-a5a1-478c-bdb7-18b03af91b72","Type":"ContainerStarted","Data":"57eb621be490be8aaa546961d39bccb19921b34898c06f5e64a080aea4ea04f5"}
Nov 03 20:56:50.423918 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:50.422068    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48" podStartSLOduration=1.422048344 podStartE2EDuration="1.422048344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:56:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:56:50.421439265 +0000 UTC m=+1012.966145638" watchObservedRunningTime="2026-11-03 20:56:50.422048344 +0000 UTC m=+1012.966754714"
Nov 03 20:56:50.521049 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:50.521015    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:51.522119 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:51.522087    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 03 20:56:51.522503 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:51.522145    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48" podUID="d549e05f-a5a1-478c-bdb7-18b03af91b72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 03 20:56:52.520614 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:52.520579    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 03 20:56:52.520803 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:52.520660    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48" podUID="d549e05f-a5a1-478c-bdb7-18b03af91b72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 03 20:56:53.521187 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:53.521149    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 03 20:56:53.521687 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:53.521656    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48" podUID="d549e05f-a5a1-478c-bdb7-18b03af91b72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 03 20:56:54.521489 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:54.521454    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 03 20:56:54.521931 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:54.521521    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48" podUID="d549e05f-a5a1-478c-bdb7-18b03af91b72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 03 20:56:55.521487 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:55.521447    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 03 20:56:55.521692 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:55.521514    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48" podUID="d549e05f-a5a1-478c-bdb7-18b03af91b72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 03 20:56:56.521449 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:56.521411    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Nov 03 20:56:56.521641 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:56.521476    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48" podUID="d549e05f-a5a1-478c-bdb7-18b03af91b72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Nov 03 20:56:57.524916 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:57.524884    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:57.525305 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:57.525127    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:56:57.525827 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:56:57.525809    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 20:57:01.672642 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:57:01.672579    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:57:05.322373 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.322328    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"]
Nov 03 20:57:05.328258 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.328222    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.381361 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.381335    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-862138-openshift-default-dockercfg-8dw8p\""
Nov 03 20:57:05.433264 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.433239    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"]
Nov 03 20:57:05.497317 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.497282    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.497702 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.497672    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.497942 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.497914    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5kgj\" (UniqueName: \"kubernetes.io/projected/732f4b2d-0d2f-400d-8b19-38c237a6ecac-kube-api-access-g5kgj\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.498308 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.498289    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.498545 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.498528    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.498771 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.498754    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.499084 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.499064    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.499347 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.499328    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.499542 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.499520    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.600692 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.600561    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.600915 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.600887    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.600981 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.600957    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.601034 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.601021    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.601086 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.601072    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.601140 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.601118    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5kgj\" (UniqueName: \"kubernetes.io/projected/732f4b2d-0d2f-400d-8b19-38c237a6ecac-kube-api-access-g5kgj\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.601194 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.601171    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.601248 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.601203    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.601248 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.601232    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.601769 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.601722    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.602052 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.602030    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.602559 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.602535    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.602687 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.602566    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.603098 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.603075    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.604878 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.604856    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.606372 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.606346    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.679064 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.679027    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.679379 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.679354    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5kgj\" (UniqueName: \"kubernetes.io/projected/732f4b2d-0d2f-400d-8b19-38c237a6ecac-kube-api-access-g5kgj\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:05.942115 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:05.942037    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:06.373428 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:06.373400    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"]
Nov 03 20:57:06.375704 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:57:06.375670    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod732f4b2d_0d2f_400d_8b19_38c237a6ecac.slice/crio-a7f9dc83097c556d08e0999b470a2bf7c8faa0b4ccbf765b0858d7b481ac67ee WatchSource:0}: Error finding container a7f9dc83097c556d08e0999b470a2bf7c8faa0b4ccbf765b0858d7b481ac67ee: Status 404 returned error can't find the container with id a7f9dc83097c556d08e0999b470a2bf7c8faa0b4ccbf765b0858d7b481ac67ee
Nov 03 20:57:06.378127 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:06.378089    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:57:06.378221 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:06.378160    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:57:06.378221 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:06.378189    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:57:06.388694 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:06.388666    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48" event={"ID":"732f4b2d-0d2f-400d-8b19-38c237a6ecac","Type":"ContainerStarted","Data":"a7f9dc83097c556d08e0999b470a2bf7c8faa0b4ccbf765b0858d7b481ac67ee"}
Nov 03 20:57:07.396008 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:07.395972    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48" event={"ID":"732f4b2d-0d2f-400d-8b19-38c237a6ecac","Type":"ContainerStarted","Data":"5755642b24d4b76bb7f641705089aefa0a166bbe35be2eb648fe43e545d667e5"}
Nov 03 20:57:07.942239 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:07.942209    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:07.943392 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:07.943365    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 03 20:57:07.943539 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:07.943433    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48" podUID="732f4b2d-0d2f-400d-8b19-38c237a6ecac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 03 20:57:08.942496 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:08.942465    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 03 20:57:08.942925 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:08.942519    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48" podUID="732f4b2d-0d2f-400d-8b19-38c237a6ecac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 03 20:57:09.943019 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:09.942980    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 03 20:57:09.943507 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:09.943041    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48" podUID="732f4b2d-0d2f-400d-8b19-38c237a6ecac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 03 20:57:10.942385 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:10.942342    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 03 20:57:10.942601 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:10.942409    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48" podUID="732f4b2d-0d2f-400d-8b19-38c237a6ecac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 03 20:57:11.942755 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:11.942722    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Nov 03 20:57:11.943145 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:11.942778    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48" podUID="732f4b2d-0d2f-400d-8b19-38c237a6ecac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Nov 03 20:57:13.147997 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:13.147956    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:13.148509 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:13.148486    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:13.149348 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:13.149327    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 20:57:13.318400 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:13.318345    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48" podStartSLOduration=8.31833277 podStartE2EDuration="8.31833277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:57:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:57:07.588354319 +0000 UTC m=+1030.133060756" watchObservedRunningTime="2026-11-03 20:57:13.31833277 +0000 UTC m=+1035.863039140"
Nov 03 20:57:27.579351 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.579321    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"]
Nov 03 20:57:27.584900 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.584864    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.660070 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.660036    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"]
Nov 03 20:57:27.681745 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.681714    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-e22abf\"/\"distinct-w5-e22abf-kserve-self-signed-certs\""
Nov 03 20:57:27.690550 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.690508    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-model-cache\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.690704 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.690557    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxk6w\" (UniqueName: \"kubernetes.io/projected/81f1a530-ef53-4782-aae4-4a8666859325-kube-api-access-kxk6w\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.690765 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.690710    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f1a530-ef53-4782-aae4-4a8666859325-tls-certs\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.690810 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.690759    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-home\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.690922 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.690902    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-dshm\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.691004 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.690989    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-tmp-dir\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.792877 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.792833    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-dshm\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.793734 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.793703    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-tmp-dir\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.794033 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.794011    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-model-cache\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.794180 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.794154    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-tmp-dir\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.794344 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.794319    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxk6w\" (UniqueName: \"kubernetes.io/projected/81f1a530-ef53-4782-aae4-4a8666859325-kube-api-access-kxk6w\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.794530 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.794505    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f1a530-ef53-4782-aae4-4a8666859325-tls-certs\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.794678 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.794362    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-model-cache\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.794678 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.794564    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-home\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.795001 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.794978    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-home\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.796227 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.796205    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-dshm\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.797178 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.797150    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f1a530-ef53-4782-aae4-4a8666859325-tls-certs\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:27.906528 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:27.906449    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxk6w\" (UniqueName: \"kubernetes.io/projected/81f1a530-ef53-4782-aae4-4a8666859325-kube-api-access-kxk6w\") pod \"distinct-w5-e22abf-kserve-dd558dcc8-2h454\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") " pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:28.203699 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:28.200851    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:28.643071 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:28.643043    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"]
Nov 03 20:57:28.645529 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:57:28.645501    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81f1a530_ef53_4782_aae4_4a8666859325.slice/crio-47409e97bb03b13632c11a30aa520e7a33c284fb9a14b26738ba1c98a0948ab0 WatchSource:0}: Error finding container 47409e97bb03b13632c11a30aa520e7a33c284fb9a14b26738ba1c98a0948ab0: Status 404 returned error can't find the container with id 47409e97bb03b13632c11a30aa520e7a33c284fb9a14b26738ba1c98a0948ab0
Nov 03 20:57:29.539797 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:29.539757    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" event={"ID":"81f1a530-ef53-4782-aae4-4a8666859325","Type":"ContainerStarted","Data":"45197fcbe777ca1d03a201d639ff5641c1dc0bf0c0bf88cbdd095b1d2d729ffd"}
Nov 03 20:57:29.539999 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:29.539809    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" event={"ID":"81f1a530-ef53-4782-aae4-4a8666859325","Type":"ContainerStarted","Data":"47409e97bb03b13632c11a30aa520e7a33c284fb9a14b26738ba1c98a0948ab0"}
Nov 03 20:57:29.661129 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:29.661079    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podStartSLOduration=2.661065691 podStartE2EDuration="2.661065691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:57:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:57:29.658920991 +0000 UTC m=+1052.203627362" watchObservedRunningTime="2026-11-03 20:57:29.661065691 +0000 UTC m=+1052.205772060"
Nov 03 20:57:30.206654 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:30.202706    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:31.669740 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:57:31.669698    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:57:38.202524 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:38.202474    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:38.216762 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:38.216739    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:38.612398 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:38.612367    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 20:57:49.947456 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:49.947419    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"]
Nov 03 20:57:49.951266 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:49.951248    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.011785 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.011753    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-home\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.011952 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.011870    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-dshm\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.011952 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.011911    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-tls-certs\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.012046 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.012009    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-tmp-dir\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.012092 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.012052    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-model-cache\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.012129 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.012112    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-525tq\" (UniqueName: \"kubernetes.io/projected/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-kube-api-access-525tq\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.061048 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.061017    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-ca4642\"/\"distinct-w4-ca4642-kserve-self-signed-certs\""
Nov 03 20:57:50.064393 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.064368    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"]
Nov 03 20:57:50.113079 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.113037    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-tmp-dir\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.113293 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.113109    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-model-cache\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.113293 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.113169    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-525tq\" (UniqueName: \"kubernetes.io/projected/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-kube-api-access-525tq\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.113293 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.113225    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-home\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.113293 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.113270    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-dshm\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.113537 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.113298    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-tls-certs\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.113617 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.113585    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-tmp-dir\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.114014 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.113989    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-home\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.114295 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.114207    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-model-cache\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.116372 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.116346    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-tls-certs\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.116462 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.116447    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-dshm\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.175212 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.175176    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-525tq\" (UniqueName: \"kubernetes.io/projected/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-kube-api-access-525tq\") pod \"distinct-w4-ca4642-kserve-68fc776df-z5k2s\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") " pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.263781 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.263743    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:50.679267 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:50.679241    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"]
Nov 03 20:57:50.680983 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:57:50.680959    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3ba8495_3500_4f1c_b81b_8fd8dd56ad3e.slice/crio-2167912ed5c2de0bb62c1ec3754005900b428ea0eb821876614de5d875176fa0 WatchSource:0}: Error finding container 2167912ed5c2de0bb62c1ec3754005900b428ea0eb821876614de5d875176fa0: Status 404 returned error can't find the container with id 2167912ed5c2de0bb62c1ec3754005900b428ea0eb821876614de5d875176fa0
Nov 03 20:57:51.152130 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.152101    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"]
Nov 03 20:57:51.155792 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.155767    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.227242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.227211    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.227242 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.227246    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.227440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.227276    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.227440 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.227418    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p859t\" (UniqueName: \"kubernetes.io/projected/68cccb32-d2a6-48f7-98f6-f0f60aab5496-kube-api-access-p859t\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.227524 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.227461    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.227561 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.227542    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.227608 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.227595    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.227682 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.227652    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.227730 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.227708    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.247339 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.247309    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"]
Nov 03 20:57:51.328353 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.328317    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p859t\" (UniqueName: \"kubernetes.io/projected/68cccb32-d2a6-48f7-98f6-f0f60aab5496-kube-api-access-p859t\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.328513 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.328366    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.328513 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.328393    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.328513 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.328417    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.328513 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.328448    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.328513 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.328465    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.328828 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.328576    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.328828 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.328602    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.328828 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.328681    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.328954 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.328889    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.329029 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.329005    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.329079 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.329017    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.329079 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.329063    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.329534 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.329517    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.331015 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.330995    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.331148 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.331129    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.389027 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.388999    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.389204 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.389184    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p859t\" (UniqueName: \"kubernetes.io/projected/68cccb32-d2a6-48f7-98f6-f0f60aab5496-kube-api-access-p859t\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.477238 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.477213    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:51.686556 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.686516    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" event={"ID":"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e","Type":"ContainerStarted","Data":"9d53d0b8a0ad644107144ce5babd9312a7ce533ee4d449a53fb25c595d6cbd7b"}
Nov 03 20:57:51.686758 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.686568    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" event={"ID":"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e","Type":"ContainerStarted","Data":"2167912ed5c2de0bb62c1ec3754005900b428ea0eb821876614de5d875176fa0"}
Nov 03 20:57:51.806901 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.806804    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podStartSLOduration=2.806789725 podStartE2EDuration="2.806789725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:57:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:57:51.804264445 +0000 UTC m=+1074.348970816" watchObservedRunningTime="2026-11-03 20:57:51.806789725 +0000 UTC m=+1074.351496094"
Nov 03 20:57:51.908061 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.908031    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"]
Nov 03 20:57:51.909378 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:57:51.909351    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68cccb32_d2a6_48f7_98f6_f0f60aab5496.slice/crio-a9c00419054eb55be50209baf68a22beb440f940682b57398abf7be237c15d7b WatchSource:0}: Error finding container a9c00419054eb55be50209baf68a22beb440f940682b57398abf7be237c15d7b: Status 404 returned error can't find the container with id a9c00419054eb55be50209baf68a22beb440f940682b57398abf7be237c15d7b
Nov 03 20:57:51.911762 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.911730    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:57:51.911842 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.911794    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:57:51.911842 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:51.911820    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:57:52.264074 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:52.264040    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:57:52.700666 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:52.700561    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m" event={"ID":"68cccb32-d2a6-48f7-98f6-f0f60aab5496","Type":"ContainerStarted","Data":"989be0d73dfc6cfdcd45f1f42c4d3197b5268e04a86043819ef67544ede6de72"}
Nov 03 20:57:52.700666 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:52.700648    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m" event={"ID":"68cccb32-d2a6-48f7-98f6-f0f60aab5496","Type":"ContainerStarted","Data":"a9c00419054eb55be50209baf68a22beb440f940682b57398abf7be237c15d7b"}
Nov 03 20:57:52.873651 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:52.871282    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m" podStartSLOduration=1.871261077 podStartE2EDuration="1.871261077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:57:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:57:52.869985888 +0000 UTC m=+1075.414692258" watchObservedRunningTime="2026-11-03 20:57:52.871261077 +0000 UTC m=+1075.415967448"
Nov 03 20:57:53.486654 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:53.478950    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:53.486654 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:53.482277    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 03 20:57:53.486654 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:53.482339    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m" podUID="68cccb32-d2a6-48f7-98f6-f0f60aab5496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 03 20:57:54.486655 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:54.486220    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 03 20:57:54.486655 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:54.486283    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m" podUID="68cccb32-d2a6-48f7-98f6-f0f60aab5496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 03 20:57:55.487667 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:55.480957    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 03 20:57:55.487667 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:55.481015    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m" podUID="68cccb32-d2a6-48f7-98f6-f0f60aab5496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 03 20:57:56.483653 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:56.483035    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 03 20:57:56.483653 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:56.483113    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m" podUID="68cccb32-d2a6-48f7-98f6-f0f60aab5496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 03 20:57:57.478669 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:57.477985    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 03 20:57:57.478669 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:57.478054    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m" podUID="68cccb32-d2a6-48f7-98f6-f0f60aab5496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 03 20:57:58.479035 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:58.478995    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Nov 03 20:57:58.479659 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:58.479602    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m" podUID="68cccb32-d2a6-48f7-98f6-f0f60aab5496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Nov 03 20:57:59.483082 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:59.483054    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:59.483537 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:59.483516    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:57:59.484205 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:57:59.484186    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 20:58:00.264971 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:00.264933    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:58:00.277873 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:00.277842    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:58:00.765031 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:00.764998    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 20:58:01.669826 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:58:01.669791    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:58:21.450702 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.450667    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"]
Nov 03 20:58:21.454565 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.454542    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.558486 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.558456    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-89bb58\"/\"distinct-2-w3-89bb58-kserve-self-signed-certs\""
Nov 03 20:58:21.560830 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.560806    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"]
Nov 03 20:58:21.606425 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.606387    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-tmp-dir\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.608763 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.608724    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-dshm\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.609261 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.608947    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7nqm\" (UniqueName: \"kubernetes.io/projected/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-kube-api-access-q7nqm\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.609261 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.609011    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-home\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.609261 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.609067    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-model-cache\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.609261 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.609107    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-tls-certs\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.710209 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.710171    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-tls-certs\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.710391 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.710223    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-tmp-dir\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.710391 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.710264    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-dshm\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.710510 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.710414    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7nqm\" (UniqueName: \"kubernetes.io/projected/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-kube-api-access-q7nqm\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.710510 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.710453    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-home\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.710510 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.710494    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-model-cache\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.710878 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.710852    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-model-cache\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.711065 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.711011    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-home\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.711326 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.711305    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-tmp-dir\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.713030 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.713000    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-dshm\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.713221 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.713200    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-tls-certs\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:21.772052 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:21.772024    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7nqm\" (UniqueName: \"kubernetes.io/projected/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-kube-api-access-q7nqm\") pod \"distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") " pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:22.066982 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:22.066879    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:22.448500 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:22.448474    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"]
Nov 03 20:58:22.450476 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:58:22.450441    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf6e552c_52c9_49be_8dc5_4c2d3dde5a2e.slice/crio-0177881eeb2d3f99fd467d4b771a377a55524991ffb85c44f753a448cb3c2385 WatchSource:0}: Error finding container 0177881eeb2d3f99fd467d4b771a377a55524991ffb85c44f753a448cb3c2385: Status 404 returned error can't find the container with id 0177881eeb2d3f99fd467d4b771a377a55524991ffb85c44f753a448cb3c2385
Nov 03 20:58:22.888772 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:22.888732    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" event={"ID":"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e","Type":"ContainerStarted","Data":"85c3797366f9f15f0b7bb5ee2452dc82578ca3f8e061878e3dc9bc6f08ad0be3"}
Nov 03 20:58:22.889263 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:22.888787    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" event={"ID":"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e","Type":"ContainerStarted","Data":"0177881eeb2d3f99fd467d4b771a377a55524991ffb85c44f753a448cb3c2385"}
Nov 03 20:58:23.013720 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:23.013280    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podStartSLOduration=2.013261031 podStartE2EDuration="2.013261031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:58:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:58:23.009993632 +0000 UTC m=+1105.554700016" watchObservedRunningTime="2026-11-03 20:58:23.013261031 +0000 UTC m=+1105.557967402"
Nov 03 20:58:23.067646 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:23.067586    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:31.670068 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:58:31.670033    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:58:32.067464 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:32.067431    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:32.081382 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:32.081358    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:32.953773 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:32.953741    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 20:58:51.190994 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.190961    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"]
Nov 03 20:58:51.194782 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.194765    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.299367 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.299335    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"]
Nov 03 20:58:51.361934 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.361907    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.362078 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.362057    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.362130 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.362117    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.362167 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.362157    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.362213 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.362179    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.362256 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.362241    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.362292 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.362267    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.362325 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.362298    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl9km\" (UniqueName: \"kubernetes.io/projected/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-kube-api-access-rl9km\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.362365 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.362349    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.462858 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.462828    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.462858 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.462860    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.463057 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.462890    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl9km\" (UniqueName: \"kubernetes.io/projected/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-kube-api-access-rl9km\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.463057 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.462919    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.463057 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.462953    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.463215 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.463090    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.463215 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.463175    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.463215 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.463205    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.463363 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.463236    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.463363 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.463266    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.463538 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.463506    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.463658 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.463578    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.463884 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.463846    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.463944 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.463928    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.465410 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.465390    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.465710 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.465690    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.523811 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.523781    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.523909 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.523870    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl9km\" (UniqueName: \"kubernetes.io/projected/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-kube-api-access-rl9km\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:51.808511 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:51.808436    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:52.239665 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:52.239641    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"]
Nov 03 20:58:52.241330 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:58:52.241300    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9e8cb2e_098a_4cd1_8d7a_af05fdcd6fa6.slice/crio-2af52f060eb52efdcb702e7625fa4c9a6df80abe1720877e68e4f03676d4f14e WatchSource:0}: Error finding container 2af52f060eb52efdcb702e7625fa4c9a6df80abe1720877e68e4f03676d4f14e: Status 404 returned error can't find the container with id 2af52f060eb52efdcb702e7625fa4c9a6df80abe1720877e68e4f03676d4f14e
Nov 03 20:58:52.243721 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:52.243686    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:58:52.243791 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:52.243745    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:58:52.243791 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:52.243774    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:58:53.067653 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:53.063572    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94" event={"ID":"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6","Type":"ContainerStarted","Data":"71538af0cfaddbaa3bf3cf6a7161b08fea108960c5f8984244cdc715f6426b22"}
Nov 03 20:58:53.067653 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:53.063653    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94" event={"ID":"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6","Type":"ContainerStarted","Data":"2af52f060eb52efdcb702e7625fa4c9a6df80abe1720877e68e4f03676d4f14e"}
Nov 03 20:58:53.236666 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:53.236583    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94" podStartSLOduration=2.236569939 podStartE2EDuration="2.236569939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:58:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:58:53.236178081 +0000 UTC m=+1135.780884486" watchObservedRunningTime="2026-11-03 20:58:53.236569939 +0000 UTC m=+1135.781276308"
Nov 03 20:58:53.808903 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:53.808864    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:53.810384 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:53.810351    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 03 20:58:53.810517 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:53.810427    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94" podUID="a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 03 20:58:54.809038 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:54.809008    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 03 20:58:54.809482 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:54.809063    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94" podUID="a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 03 20:58:55.808964 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:55.808928    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 03 20:58:55.809140 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:55.808973    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94" podUID="a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 03 20:58:56.809120 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:56.809090    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 03 20:58:56.809293 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:56.809138    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94" podUID="a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 03 20:58:57.809296 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:57.809265    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 03 20:58:57.809699 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:57.809316    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94" podUID="a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 03 20:58:58.809574 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:58.809532    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Nov 03 20:58:58.810052 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:58.809594    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94" podUID="a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Nov 03 20:58:59.815293 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:59.815255    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:59.815849 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:59.815819    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:58:59.816593 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:58:59.816568    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 20:59:01.670108 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:59:01.670071    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:59:20.105986 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.105954    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"]
Nov 03 20:59:20.109644 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.109605    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.126900 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.126874    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crjj5\" (UniqueName: \"kubernetes.io/projected/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-kube-api-access-crjj5\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.127030 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.126921    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.127541 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.127504    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.127689 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.127588    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.127745 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.127721    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.127780 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.127753    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.127886 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.127858    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.127926 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.127909    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.127962 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.127946    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.213831 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.213799    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"]
Nov 03 20:59:20.229341 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.229295    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crjj5\" (UniqueName: \"kubernetes.io/projected/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-kube-api-access-crjj5\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.229504 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.229352    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.229504 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.229423    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.229504 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.229449    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.229504 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.229491    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.230114 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.229513    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.230114 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.229562    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.230114 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.229593    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.230114 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.229643    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.230114 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.229865    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.230114 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.230110    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.230424 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.230185    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.230481 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.230424    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.230997 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.230973    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.232828 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.232807    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.233178 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.233139    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.290787 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.290754    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.290787 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.290777    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crjj5\" (UniqueName: \"kubernetes.io/projected/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-kube-api-access-crjj5\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.426010 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.425945    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:20.859491 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.859460    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"]
Nov 03 20:59:20.861809 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:59:20.861778    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb50b6fe_df28_4b1d_8a77_f7cdecf833c7.slice/crio-45b1d181d8ac540e11d9ff76bb8ef08062b6f24ebcae675809f016ab017496c0 WatchSource:0}: Error finding container 45b1d181d8ac540e11d9ff76bb8ef08062b6f24ebcae675809f016ab017496c0: Status 404 returned error can't find the container with id 45b1d181d8ac540e11d9ff76bb8ef08062b6f24ebcae675809f016ab017496c0
Nov 03 20:59:20.864085 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.864051    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:59:20.864191 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.864118    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:59:20.864191 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:20.864149    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:59:21.285641 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:21.285583    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw" event={"ID":"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7","Type":"ContainerStarted","Data":"7f86e68a1e3a1d14f75ecdd4554ba453fc4fdb4a5ce86b2e4ef9928d5137c87c"}
Nov 03 20:59:21.286174 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:21.285665    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw" event={"ID":"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7","Type":"ContainerStarted","Data":"45b1d181d8ac540e11d9ff76bb8ef08062b6f24ebcae675809f016ab017496c0"}
Nov 03 20:59:21.429839 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:21.429794    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:21.431441 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:21.431415    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 03 20:59:21.431592 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:21.431467    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw" podUID="eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 03 20:59:21.455007 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:21.454949    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw" podStartSLOduration=1.454932405 podStartE2EDuration="1.454932405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:59:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:59:21.454414763 +0000 UTC m=+1163.999121132" watchObservedRunningTime="2026-11-03 20:59:21.454932405 +0000 UTC m=+1163.999638775"
Nov 03 20:59:22.426586 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:22.426553    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 03 20:59:22.426989 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:22.426608    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw" podUID="eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 03 20:59:23.426993 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:23.426959    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 03 20:59:23.427395 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:23.427010    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw" podUID="eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 03 20:59:24.426279 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:24.426243    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 03 20:59:24.426496 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:24.426307    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw" podUID="eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 03 20:59:25.426478 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:25.426442    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 03 20:59:25.426860 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:25.426501    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw" podUID="eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 03 20:59:26.427153 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:26.427122    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 03 20:59:26.427543 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:26.427174    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw" podUID="eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 03 20:59:27.426432 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:27.426399    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Nov 03 20:59:27.426596 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:27.426449    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw" podUID="eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Nov 03 20:59:28.430988 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:28.430949    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:28.431453 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:28.431348    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:28.431973 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:28.431952    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 20:59:31.672623 ip-10-0-142-223 kubenswrapper[4993]: E1103 20:59:31.672579    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 20:59:37.006169 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.006078    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"]
Nov 03 20:59:37.011750 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.011726    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.156396 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.156349    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.156613 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.156411    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.156613 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.156503    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.156613 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.156561    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.156848 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.156666    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.156848 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.156707    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.156848 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.156736    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.156848 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.156790    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf9kt\" (UniqueName: \"kubernetes.io/projected/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-kube-api-access-mf9kt\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.156848 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.156818    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.247213 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.247179    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"]
Nov 03 20:59:37.258489 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.258411    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf9kt\" (UniqueName: \"kubernetes.io/projected/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-kube-api-access-mf9kt\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.258489 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.258468    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.258697 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.258515    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.258697 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.258535    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.258697 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.258600    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.258697 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.258660    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.258909 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.258718    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.258909 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.258746    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.258909 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.258763    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.259442 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.259421    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-credential-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.260095 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.260034    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-workload-socket\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.260330 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.260306    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-workload-certs\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.260594 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.260570    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-data\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.260705 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.260674    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istiod-ca-cert\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.263301 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.263278    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-envoy\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.263388 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.263372    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-podinfo\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.329215 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.329180    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf9kt\" (UniqueName: \"kubernetes.io/projected/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-kube-api-access-mf9kt\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.329679 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.329658    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-token\") pod \"e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") " pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.346127 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.346106    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:37.789657 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.788751    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"]
Nov 03 20:59:37.799099 ip-10-0-142-223 kubenswrapper[4993]: W1103 20:59:37.798061    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod529e3166_0d8e_40e1_93a7_a6b563ce5c8c.slice/crio-60c2e91d1b1e5c0e03a87e2d7d4f3f5568eeb1ce3e691129f1b71bcd5ca46b34 WatchSource:0}: Error finding container 60c2e91d1b1e5c0e03a87e2d7d4f3f5568eeb1ce3e691129f1b71bcd5ca46b34: Status 404 returned error can't find the container with id 60c2e91d1b1e5c0e03a87e2d7d4f3f5568eeb1ce3e691129f1b71bcd5ca46b34
Nov 03 20:59:37.801851 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.801815    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:59:37.801963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.801901    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:59:37.801963 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:37.801949    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 20:59:38.410404 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:38.410363    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m" event={"ID":"529e3166-0d8e-40e1-93a7-a6b563ce5c8c","Type":"ContainerStarted","Data":"0562b913997b3c2ef80cc31808bce032050be81a3704bcecd3f6789395f1b200"}
Nov 03 20:59:38.410898 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:38.410426    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m" event={"ID":"529e3166-0d8e-40e1-93a7-a6b563ce5c8c","Type":"ContainerStarted","Data":"60c2e91d1b1e5c0e03a87e2d7d4f3f5568eeb1ce3e691129f1b71bcd5ca46b34"}
Nov 03 20:59:38.637824 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:38.637776    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m" podStartSLOduration=2.637762312 podStartE2EDuration="2.637762312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 20:59:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 20:59:38.586118633 +0000 UTC m=+1181.130825003" watchObservedRunningTime="2026-11-03 20:59:38.637762312 +0000 UTC m=+1181.182468681"
Nov 03 20:59:39.346701 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:39.346665    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:39.348405 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:39.348375    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 20:59:39.348553 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:39.348435    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m" podUID="529e3166-0d8e-40e1-93a7-a6b563ce5c8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 20:59:40.347186 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:40.347140    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 20:59:40.347732 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:40.347200    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m" podUID="529e3166-0d8e-40e1-93a7-a6b563ce5c8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 20:59:41.346657 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:41.346609    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 20:59:41.346833 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:41.346685    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m" podUID="529e3166-0d8e-40e1-93a7-a6b563ce5c8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 20:59:42.346916 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:42.346885    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 20:59:42.347374 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:42.346944    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m" podUID="529e3166-0d8e-40e1-93a7-a6b563ce5c8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 20:59:43.346564 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:43.346533    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 20:59:43.346807 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:43.346584    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m" podUID="529e3166-0d8e-40e1-93a7-a6b563ce5c8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 20:59:44.346721 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:44.346683    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Nov 03 20:59:44.347218 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:44.346744    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m" podUID="529e3166-0d8e-40e1-93a7-a6b563ce5c8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Nov 03 20:59:45.351323 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:45.351292    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:45.351788 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:45.351720    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 20:59:45.352393 ip-10-0-142-223 kubenswrapper[4993]: I1103 20:59:45.352377    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 21:00:00.538659 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.538608    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"]
Nov 03 21:00:00.542971 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.542946    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.602401 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.602370    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"]
Nov 03 21:00:00.647647 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.647607    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-499187\"/\"distinct-2-w0-499187-kserve-self-signed-certs\""
Nov 03 21:00:00.651744 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.651718    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-dshm\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.651871 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.651774    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-home\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.651871 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.651799    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5af5bf28-fe61-4653-b16a-8640b3156119-tls-certs\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.651871 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.651856    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qdkm\" (UniqueName: \"kubernetes.io/projected/5af5bf28-fe61-4653-b16a-8640b3156119-kube-api-access-9qdkm\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.652132 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.652113    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-tmp-dir\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.652202 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.652177    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-model-cache\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.753563 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.753517    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-tmp-dir\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.753799 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.753608    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-model-cache\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.753799 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.753707    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-dshm\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.753799 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.753761    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-home\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.753799 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.753784    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5af5bf28-fe61-4653-b16a-8640b3156119-tls-certs\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.754011 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.753809    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qdkm\" (UniqueName: \"kubernetes.io/projected/5af5bf28-fe61-4653-b16a-8640b3156119-kube-api-access-9qdkm\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.754990 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.754952    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-model-cache\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.755281 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.755257    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-tmp-dir\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.755489 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.755403    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-home\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.757782 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.757741    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-dshm\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.757880 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.757816    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5af5bf28-fe61-4653-b16a-8640b3156119-tls-certs\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.816096 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.816016    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qdkm\" (UniqueName: \"kubernetes.io/projected/5af5bf28-fe61-4653-b16a-8640b3156119-kube-api-access-9qdkm\") pod \"distinct-2-w0-499187-kserve-56fd64468b-w9m8b\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") " pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:00.854952 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:00.854915    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:01.309519 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:01.309488    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"]
Nov 03 21:00:01.311039 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:00:01.311004    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5af5bf28_fe61_4653_b16a_8640b3156119.slice/crio-73ada115ed9f98ef42b8c85a41aa16e7714948c3dc7725fc12bfd7f32db9449b WatchSource:0}: Error finding container 73ada115ed9f98ef42b8c85a41aa16e7714948c3dc7725fc12bfd7f32db9449b: Status 404 returned error can't find the container with id 73ada115ed9f98ef42b8c85a41aa16e7714948c3dc7725fc12bfd7f32db9449b
Nov 03 21:00:01.546968 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:01.546929    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" event={"ID":"5af5bf28-fe61-4653-b16a-8640b3156119","Type":"ContainerStarted","Data":"0481e49b495dbd704128c4836ce10dac0842da047b1820ddfebff8fd85370338"}
Nov 03 21:00:01.547418 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:01.546976    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" event={"ID":"5af5bf28-fe61-4653-b16a-8640b3156119","Type":"ContainerStarted","Data":"73ada115ed9f98ef42b8c85a41aa16e7714948c3dc7725fc12bfd7f32db9449b"}
Nov 03 21:00:01.665782 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:01.665682    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podStartSLOduration=1.665668202 podStartE2EDuration="1.665668202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:00:01.662536744 +0000 UTC m=+1204.207243113" watchObservedRunningTime="2026-11-03 21:00:01.665668202 +0000 UTC m=+1204.210374570"
Nov 03 21:00:01.669830 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:00:01.669802    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:00:01.855058 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:01.855024    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:10.855373 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:10.855337    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:10.867576 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:10.867536    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:11.639126 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:11.639092    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:30.871533 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:30.871492    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"]
Nov 03 21:00:30.875502 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:30.875481    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:30.972349 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:30.972310    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-dshm\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:30.972596 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:30.972577    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-tmp-dir\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:30.973812 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:30.973785    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-model-cache\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:30.973943 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:30.973840    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-home\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:30.974023 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:30.974006    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49gxw\" (UniqueName: \"kubernetes.io/projected/8fe2fef6-f3ea-49a2-aa27-47635359b017-kube-api-access-49gxw\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:30.974081 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:30.974048    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe2fef6-f3ea-49a2-aa27-47635359b017-tls-certs\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:30.979882 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:30.979855    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-862138\"/\"distinct-2-w2-862138-kserve-self-signed-certs\""
Nov 03 21:00:30.983750 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:30.983725    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"]
Nov 03 21:00:31.075016 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.074981    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49gxw\" (UniqueName: \"kubernetes.io/projected/8fe2fef6-f3ea-49a2-aa27-47635359b017-kube-api-access-49gxw\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:31.075203 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.075035    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe2fef6-f3ea-49a2-aa27-47635359b017-tls-certs\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:31.075203 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.075165    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-dshm\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:31.075203 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.075192    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-tmp-dir\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:31.076052 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.075256    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-model-cache\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:31.076246 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.076228    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-home\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:31.076719 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.076107    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-model-cache\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:31.076719 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.076686    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-tmp-dir\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:31.078416 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.078386    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-home\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:31.079388 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.079366    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-dshm\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:31.079871 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.079826    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe2fef6-f3ea-49a2-aa27-47635359b017-tls-certs\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:31.137878 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.137798    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49gxw\" (UniqueName: \"kubernetes.io/projected/8fe2fef6-f3ea-49a2-aa27-47635359b017-kube-api-access-49gxw\") pod \"distinct-2-w2-862138-kserve-849dbc6564-rlh78\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") " pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:31.188668 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.188614    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:31.589235 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.589196    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"]
Nov 03 21:00:31.589479 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:00:31.589446    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8fe2fef6_f3ea_49a2_aa27_47635359b017.slice/crio-fba58e00fd0265cc97fb4c3f5d244bb1337a41eb5f91ba493ea35c03c27f14fa WatchSource:0}: Error finding container fba58e00fd0265cc97fb4c3f5d244bb1337a41eb5f91ba493ea35c03c27f14fa: Status 404 returned error can't find the container with id fba58e00fd0265cc97fb4c3f5d244bb1337a41eb5f91ba493ea35c03c27f14fa
Nov 03 21:00:31.673595 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:00:31.673564    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:00:31.748970 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.748938    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" event={"ID":"8fe2fef6-f3ea-49a2-aa27-47635359b017","Type":"ContainerStarted","Data":"aa03008a9ce10d11ac59d7299076fb98bdc5dea3722df8f90485ec1057531163"}
Nov 03 21:00:31.749155 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.748982    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" event={"ID":"8fe2fef6-f3ea-49a2-aa27-47635359b017","Type":"ContainerStarted","Data":"fba58e00fd0265cc97fb4c3f5d244bb1337a41eb5f91ba493ea35c03c27f14fa"}
Nov 03 21:00:31.870332 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:31.870228    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podStartSLOduration=1.8702126159999999 podStartE2EDuration="1.870212616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:00:31.866977848 +0000 UTC m=+1234.411684219" watchObservedRunningTime="2026-11-03 21:00:31.870212616 +0000 UTC m=+1234.414918985"
Nov 03 21:00:32.189740 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:32.189669    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:35.270102 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.270068    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"]
Nov 03 21:00:35.274407 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.274381    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.326606 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.326570    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-ca4642-openshift-default-dockercfg-gbv98\""
Nov 03 21:00:35.336402 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.336362    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-istio-data\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.336544 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.336408    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76814bbc-dc45-440a-947c-1f4979ad86de-istiod-ca-cert\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.336544 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.336491    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76814bbc-dc45-440a-947c-1f4979ad86de-istio-podinfo\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.336544 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.336521    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fr4r\" (UniqueName: \"kubernetes.io/projected/76814bbc-dc45-440a-947c-1f4979ad86de-kube-api-access-4fr4r\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.336679 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.336581    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-credential-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.336679 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.336653    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76814bbc-dc45-440a-947c-1f4979ad86de-istio-token\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.336756 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.336727    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-istio-envoy\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.336756 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.336750    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-workload-certs\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.336850 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.336834    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-workload-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.380298 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.380266    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"]
Nov 03 21:00:35.437909 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.437877    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76814bbc-dc45-440a-947c-1f4979ad86de-istio-token\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.438108 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.437975    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-istio-envoy\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.438108 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.438005    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-workload-certs\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.438108 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.438037    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-workload-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.438292 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.438144    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-istio-data\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.438292 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.438176    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76814bbc-dc45-440a-947c-1f4979ad86de-istiod-ca-cert\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.438411 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.438317    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76814bbc-dc45-440a-947c-1f4979ad86de-istio-podinfo\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.438411 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.438355    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fr4r\" (UniqueName: \"kubernetes.io/projected/76814bbc-dc45-440a-947c-1f4979ad86de-kube-api-access-4fr4r\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.438718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.438414    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-credential-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.438718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.438466    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-workload-certs\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.438718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.438536    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-istio-data\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.438718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.438560    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-workload-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.438943 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.438927    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-credential-socket\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.439131 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.439111    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76814bbc-dc45-440a-947c-1f4979ad86de-istiod-ca-cert\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.440528 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.440506    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-istio-envoy\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.441049 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.441030    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76814bbc-dc45-440a-947c-1f4979ad86de-istio-podinfo\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.499782 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.499754    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fr4r\" (UniqueName: \"kubernetes.io/projected/76814bbc-dc45-440a-947c-1f4979ad86de-kube-api-access-4fr4r\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.499895 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.499867    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76814bbc-dc45-440a-947c-1f4979ad86de-istio-token\") pod \"e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") " pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:35.591871 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:35.591801    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:36.025304 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.025266    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"]
Nov 03 21:00:36.030534 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.030513    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.031782 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:00:36.031759    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76814bbc_dc45_440a_947c_1f4979ad86de.slice/crio-bf329afd71390f631502085c216e65f3f0d8976a3f16596372ca6f2cb91d3a3d WatchSource:0}: Error finding container bf329afd71390f631502085c216e65f3f0d8976a3f16596372ca6f2cb91d3a3d: Status 404 returned error can't find the container with id bf329afd71390f631502085c216e65f3f0d8976a3f16596372ca6f2cb91d3a3d
Nov 03 21:00:36.034589 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.034560    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:36.034669 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.034620    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:36.034726 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.034669    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:36.075798 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.075771    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"]
Nov 03 21:00:36.126521 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.126492    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"]
Nov 03 21:00:36.147664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.146414    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.147664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.146496    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.147664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.146552    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.147664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.146756    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.147664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.146791    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7djw9\" (UniqueName: \"kubernetes.io/projected/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-kube-api-access-7djw9\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.147664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.146826    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.147664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.146962    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.147664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.147013    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.147664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.147043    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.248222 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.248185    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.248383 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.248253    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.248383 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.248284    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.248383 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.248339    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.248491 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.248408    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.248562 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.248546    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.248688 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.248672    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.249029 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.248701    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7djw9\" (UniqueName: \"kubernetes.io/projected/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-kube-api-access-7djw9\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.249155 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.249056    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.249155 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.248881    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.249155 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.248761    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.249155 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.249066    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.249392 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.249175    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.249392 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.249360    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.250988 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.250962    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.251570 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.251543    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.309882 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.309816    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7djw9\" (UniqueName: \"kubernetes.io/projected/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-kube-api-access-7djw9\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.309882 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.309817    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.352104 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.352069    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:36.373519 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.373488    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"]
Nov 03 21:00:36.378549 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.378527    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.414920 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.414881    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"]
Nov 03 21:00:36.419749 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.419725    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.462893 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.453281    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.462893 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.453330    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.462893 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.453359    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.462893 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.453388    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkj2b\" (UniqueName: \"kubernetes.io/projected/0ceb5aa1-a956-4442-b710-36453c7a1e79-kube-api-access-qkj2b\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.462893 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.453494    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ceb5aa1-a956-4442-b710-36453c7a1e79-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.462893 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.453569    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.462893 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.453601    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.462893 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.453622    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.462893 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.453717    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.462893 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.456025    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"]
Nov 03 21:00:36.484575 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.483569    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"]
Nov 03 21:00:36.555364 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.555330    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.555364 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.555371    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.555641 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.555394    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.555641 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.555424    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkj2b\" (UniqueName: \"kubernetes.io/projected/0ceb5aa1-a956-4442-b710-36453c7a1e79-kube-api-access-qkj2b\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.555641 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.555453    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.555641 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.555486    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.555641 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.555508    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.555851 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.555732    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ceb5aa1-a956-4442-b710-36453c7a1e79-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.555851 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.555760    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.555922 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.555893    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.555957 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.555942    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.555995 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.555973    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.556030 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.556004    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.556068 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.556035    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.556101 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.556064    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.556154 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.556134    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.556191 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.556182    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdn6c\" (UniqueName: \"kubernetes.io/projected/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-kube-api-access-vdn6c\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.556238 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.556221    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.556271 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.556256    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.556320 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.556302    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ceb5aa1-a956-4442-b710-36453c7a1e79-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.556510 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.556494    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.556706 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.556683    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.556908 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.556883    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.558173 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.558152    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.558656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.558603    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.617245 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.617171    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.617379 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.617301    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkj2b\" (UniqueName: \"kubernetes.io/projected/0ceb5aa1-a956-4442-b710-36453c7a1e79-kube-api-access-qkj2b\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.664898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.661208    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.664898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.661265    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.664898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.661300    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.664898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.661400    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.664898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.661442    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdn6c\" (UniqueName: \"kubernetes.io/projected/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-kube-api-access-vdn6c\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.664898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.661518    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.664898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.661591    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.664898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.661646    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.664898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.661671    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.664898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.662673    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istiod-ca-cert\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.664898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.663478    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-credential-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.664898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.663769    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-workload-socket\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.667764 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.667425    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-workload-certs\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.667764 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.667714    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-data\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.668221 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.668180    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-envoy\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.675303 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.673156    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-podinfo\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.696898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.696803    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:36.727186 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.725264    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-token\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.727186 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.727138    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdn6c\" (UniqueName: \"kubernetes.io/projected/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-kube-api-access-vdn6c\") pod \"e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") " pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.745312 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.745279    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"]
Nov 03 21:00:36.745535 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:00:36.745425    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod298f7d43_f4d7_4108_b8a1_8df13e36ccc9.slice/crio-a32ef313a3a19ed2ff6b3b074e9605fa1e50a1a77bc2809205dab3554a6a9b51 WatchSource:0}: Error finding container a32ef313a3a19ed2ff6b3b074e9605fa1e50a1a77bc2809205dab3554a6a9b51: Status 404 returned error can't find the container with id a32ef313a3a19ed2ff6b3b074e9605fa1e50a1a77bc2809205dab3554a6a9b51
Nov 03 21:00:36.749039 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.749005    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:36.749146 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.749075    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:36.749146 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.749105    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:36.808658 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.807913    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:36.819336 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.819063    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f" event={"ID":"76814bbc-dc45-440a-947c-1f4979ad86de","Type":"ContainerStarted","Data":"b483504c16bb5cd44be0e00309d1b16e4e0d18147b79590a4956ea706f8da13d"}
Nov 03 21:00:36.819336 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.819125    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f" event={"ID":"76814bbc-dc45-440a-947c-1f4979ad86de","Type":"ContainerStarted","Data":"bf329afd71390f631502085c216e65f3f0d8976a3f16596372ca6f2cb91d3a3d"}
Nov 03 21:00:36.821681 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.821223    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t" event={"ID":"298f7d43-f4d7-4108-b8a1-8df13e36ccc9","Type":"ContainerStarted","Data":"a32ef313a3a19ed2ff6b3b074e9605fa1e50a1a77bc2809205dab3554a6a9b51"}
Nov 03 21:00:36.951224 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:36.951164    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f" podStartSLOduration=1.951143751 podStartE2EDuration="1.951143751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:00:36.947789135 +0000 UTC m=+1239.492495506" watchObservedRunningTime="2026-11-03 21:00:36.951143751 +0000 UTC m=+1239.495850120"
Nov 03 21:00:37.123466 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.123433    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"]
Nov 03 21:00:37.124371 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:00:37.124323    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ceb5aa1_a956_4442_b710_36453c7a1e79.slice/crio-5a5cdf4e927eace42dd816e80f6b3618ba49a31739e647a2e134f0c5d6b012a1 WatchSource:0}: Error finding container 5a5cdf4e927eace42dd816e80f6b3618ba49a31739e647a2e134f0c5d6b012a1: Status 404 returned error can't find the container with id 5a5cdf4e927eace42dd816e80f6b3618ba49a31739e647a2e134f0c5d6b012a1
Nov 03 21:00:37.127565 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.127529    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:37.127702 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.127596    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:37.127702 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.127641    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:37.247019 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:00:37.246982    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod996b6dcf_8250_4c0e_8144_b8cbb85a2b25.slice/crio-ce2309ea8eb1deeebd0b6e71bbe20f9fb6fddcc26b11c5589fcead01b4993314 WatchSource:0}: Error finding container ce2309ea8eb1deeebd0b6e71bbe20f9fb6fddcc26b11c5589fcead01b4993314: Status 404 returned error can't find the container with id ce2309ea8eb1deeebd0b6e71bbe20f9fb6fddcc26b11c5589fcead01b4993314
Nov 03 21:00:37.248288 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.248258    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"]
Nov 03 21:00:37.250398 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.250364    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:37.250498 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.250478    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:37.250547 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.250532    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:00:37.633659 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.607213    4993 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 21:00:37.633659 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.607275    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f" podUID="76814bbc-dc45-440a-947c-1f4979ad86de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 21:00:37.633659 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.611141    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:37.844982 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.844951    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t" event={"ID":"298f7d43-f4d7-4108-b8a1-8df13e36ccc9","Type":"ContainerStarted","Data":"f3f07d04f2f506a38a084245252bb375517ec0dc3ab33de427e1622338a1fb66"}
Nov 03 21:00:37.847440 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.847415    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" event={"ID":"996b6dcf-8250-4c0e-8144-b8cbb85a2b25","Type":"ContainerStarted","Data":"5fd160bdd9e28a134219553c915944b93f4f567aeffcf1aaee966416153cd931"}
Nov 03 21:00:37.847615 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.847601    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" event={"ID":"996b6dcf-8250-4c0e-8144-b8cbb85a2b25","Type":"ContainerStarted","Data":"ce2309ea8eb1deeebd0b6e71bbe20f9fb6fddcc26b11c5589fcead01b4993314"}
Nov 03 21:00:37.850205 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.850186    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" event={"ID":"0ceb5aa1-a956-4442-b710-36453c7a1e79","Type":"ContainerStarted","Data":"dd5b31adfc0ad4feb45a7383ace9f9f20737d7b81f710c0df4436f01821f539a"}
Nov 03 21:00:37.850341 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:37.850330    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" event={"ID":"0ceb5aa1-a956-4442-b710-36453c7a1e79","Type":"ContainerStarted","Data":"5a5cdf4e927eace42dd816e80f6b3618ba49a31739e647a2e134f0c5d6b012a1"}
Nov 03 21:00:38.118551 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:38.118464    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t" podStartSLOduration=3.118443359 podStartE2EDuration="3.118443359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:00:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:00:37.980001014 +0000 UTC m=+1240.524707379" watchObservedRunningTime="2026-11-03 21:00:38.118443359 +0000 UTC m=+1240.663149732"
Nov 03 21:00:38.359514 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:38.359481    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:38.383033 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:38.382809    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" podStartSLOduration=2.382785579 podStartE2EDuration="2.382785579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:00:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:00:38.327591621 +0000 UTC m=+1240.872297994" watchObservedRunningTime="2026-11-03 21:00:38.382785579 +0000 UTC m=+1240.927491949"
Nov 03 21:00:38.384737 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:38.384618    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" podStartSLOduration=2.384512108 podStartE2EDuration="2.384512108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:00:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:00:38.122290797 +0000 UTC m=+1240.666997167" watchObservedRunningTime="2026-11-03 21:00:38.384512108 +0000 UTC m=+1240.929218481"
Nov 03 21:00:38.387308 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:38.387282    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 03 21:00:38.387505 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:38.387481    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t" podUID="298f7d43-f4d7-4108-b8a1-8df13e36ccc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 03 21:00:38.593155 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:38.593122    4993 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 21:00:38.593472 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:38.593432    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f" podUID="76814bbc-dc45-440a-947c-1f4979ad86de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 21:00:38.699974 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:38.699884    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:38.700604 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:38.700576    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 03 21:00:38.700808 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:38.700779    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" podUID="0ceb5aa1-a956-4442-b710-36453c7a1e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 03 21:00:38.811821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:38.811788    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:38.812173 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:38.812149    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 03 21:00:38.812348 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:38.812321    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" podUID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 03 21:00:39.240774 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.240724    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:39.357148 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.357092    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 03 21:00:39.357475 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.357420    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t" podUID="298f7d43-f4d7-4108-b8a1-8df13e36ccc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 03 21:00:39.359743 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.359706    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:39.609725 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.588813    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:39.609725 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.592982    4993 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 21:00:39.609725 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.593043    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f" podUID="76814bbc-dc45-440a-947c-1f4979ad86de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 21:00:39.609725 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.605707    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:39.640657 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.635735    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:39.656655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.655831    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:39.714656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.706474    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 03 21:00:39.714656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.706531    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" podUID="0ceb5aa1-a956-4442-b710-36453c7a1e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 03 21:00:39.736653 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.727898    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:39.768662 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.765712    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:39.812666 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.810029    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 03 21:00:39.812666 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.810089    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" podUID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 03 21:00:39.889734 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.885732    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:39.907664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.905723    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 21:00:39.959684 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:39.956918    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:40.084651 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.083748    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:40.362821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.362759    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 03 21:00:40.363019 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.362824    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t" podUID="298f7d43-f4d7-4108-b8a1-8df13e36ccc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 03 21:00:40.385662 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.377767    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:40.385662 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.378228    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 21:00:40.473656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.472712    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/health\": context deadline exceeded"
Nov 03 21:00:40.604656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.604073    4993 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 21:00:40.604656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.604140    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f" podUID="76814bbc-dc45-440a-947c-1f4979ad86de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 21:00:40.604656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.604233    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:40.604656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.604564    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 21:00:40.624999 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.614869    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:40.624999 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.615203    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 21:00:40.747655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.722282    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 03 21:00:40.747655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.722651    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:40.786656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.757295    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 03 21:00:40.790654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.786929    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" podUID="0ceb5aa1-a956-4442-b710-36453c7a1e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 03 21:00:40.815738 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.796071    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 03 21:00:40.817679 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.817653    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 21:00:40.833665 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.833532    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Nov 03 21:00:40.850532 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.834193    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 21:00:40.850532 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.850241    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 03 21:00:40.850532 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.850306    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" podUID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 03 21:00:40.899762 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.886508    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:40.899762 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.886907    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 21:00:40.960662 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.916527    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:40.960662 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.916967    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 21:00:40.986655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.961538    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Nov 03 21:00:40.986655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.962013    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 21:00:40.991732 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.991691    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:40.994572 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:40.994546    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 21:00:41.019575 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.015718    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:41.052501 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.052471    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 21:00:41.182567 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.147976    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:41.182567 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.148338    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 21:00:41.239654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.214690    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:41.298654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.273576    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:41.408659 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.358535    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 03 21:00:41.408659 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.358596    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t" podUID="298f7d43-f4d7-4108-b8a1-8df13e36ccc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 03 21:00:41.408659 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.378794    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:41.424981 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.424946    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:41.618807 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.618754    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:41.665732 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.664120    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Nov 03 21:00:41.786723 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.739993    4993 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 21:00:41.786723 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.740077    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f" podUID="76814bbc-dc45-440a-947c-1f4979ad86de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 21:00:41.786723 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.766896    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:41.829742 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.829460    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:41.836764 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.836722    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Nov 03 21:00:41.861655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.853524    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 03 21:00:41.861655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.853585    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" podUID="0ceb5aa1-a956-4442-b710-36453c7a1e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 03 21:00:41.895900 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.895785    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 03 21:00:41.895900 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.895853    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" podUID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 03 21:00:41.896128 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.895919    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": context deadline exceeded"
Nov 03 21:00:41.896128 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.895984    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:41.933686 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.922761    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:41.970656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.961136    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 21:00:41.970656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:41.969303    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:42.028656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.027766    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:42.040656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.038026    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:42.040656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.039737    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 21:00:42.149715 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.148916    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:42.373651 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.372955    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 03 21:00:42.373651 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.373032    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t" podUID="298f7d43-f4d7-4108-b8a1-8df13e36ccc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 03 21:00:42.412760 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.387382    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:42.620994 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.620955    4993 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 21:00:42.621816 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.621782    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f" podUID="76814bbc-dc45-440a-947c-1f4979ad86de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 21:00:42.623216 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.623187    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:42.833756 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.833615    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:42.869921 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.844224    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:42.871112 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.845093    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Nov 03 21:00:42.871448 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.848796    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 03 21:00:42.871448 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.871355    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" podUID="0ceb5aa1-a956-4442-b710-36453c7a1e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 03 21:00:42.871448 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.869056    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 03 21:00:42.871448 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.871421    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" podUID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 03 21:00:42.871845 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.871812    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Nov 03 21:00:42.897521 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.896780    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:43.097723 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.961701    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:43.097723 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:42.978585    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:43.097723 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.038203    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:43.097723 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.060729    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:43.101650 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.098863    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:43.215538 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.194714    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:43.215538 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.195034    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 03 21:00:43.363712 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.359745    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/health\": context deadline exceeded"
Nov 03 21:00:43.530697 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.360701    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 03 21:00:43.532589 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.514303    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 03 21:00:43.532827 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.532783    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t" podUID="298f7d43-f4d7-4108-b8a1-8df13e36ccc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 03 21:00:43.743845 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.688947    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": context deadline exceeded"
Nov 03 21:00:43.743845 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.710224    4993 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 21:00:43.743845 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.710280    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f" podUID="76814bbc-dc45-440a-947c-1f4979ad86de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 21:00:43.743845 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.739794    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 03 21:00:43.743845 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.739865    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" podUID="0ceb5aa1-a956-4442-b710-36453c7a1e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 03 21:00:43.748394 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.748364    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:00:43.825662 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.814032    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 03 21:00:43.825662 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.814111    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" podUID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 03 21:00:43.850183 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.850148    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 21:00:43.876664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.872758    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:43.920446 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.920391    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:43.966823 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:43.966771    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Nov 03 21:00:44.011706 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.008209    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Nov 03 21:00:44.011706 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.008525    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:44.011706 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.008844    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 21:00:44.071672 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.054990    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:44.089670 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.073765    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 21:00:44.108667 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.097548    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 21:00:44.221666 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.218172    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:44.221666 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.219329    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 03 21:00:44.399071 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.398958    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 03 21:00:44.399338 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.399289    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t" podUID="298f7d43-f4d7-4108-b8a1-8df13e36ccc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 03 21:00:44.454970 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.454937    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 21:00:44.657740 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.654832    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:44.698652 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.691782    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:44.699187 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.699098    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 03 21:00:44.699187 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.699143    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" podUID="0ceb5aa1-a956-4442-b710-36453c7a1e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 03 21:00:44.703654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.699574    4993 patch_prober.go:29] interesting pod/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Nov 03 21:00:44.703844 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.703822    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f" podUID="76814bbc-dc45-440a-947c-1f4979ad86de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Nov 03 21:00:44.848982 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.848942    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 03 21:00:44.959973 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.860659    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 03 21:00:44.959973 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.952736    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" podUID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 03 21:00:44.959973 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:44.863732    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:45.037754 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.024555    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": context deadline exceeded"
Nov 03 21:00:45.038419 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.038377    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Nov 03 21:00:45.039183 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.039095    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 21:00:45.065857 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.065832    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:00:45.109283 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.109243    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 21:00:45.115779 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.115731    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:45.117313 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.117290    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:45.221028 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.220936    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:45.309681 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.302924    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 21:00:45.320313 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.320283    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 21:00:45.356681 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.356651    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 21:00:45.359610 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.359582    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body=
Nov 03 21:00:45.364244 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.364201    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t" podUID="298f7d43-f4d7-4108-b8a1-8df13e36ccc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused"
Nov 03 21:00:45.371669 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.364070    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 21:00:45.558341 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.558222    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 21:00:45.560480 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.560452    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 21:00:45.562268 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.562069    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 21:00:45.562268 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.562157    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 21:00:45.709816 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.709782    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Nov 03 21:00:45.710114 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.710063    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" podUID="0ceb5aa1-a956-4442-b710-36453c7a1e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Nov 03 21:00:45.826869 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.826778    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Nov 03 21:00:45.827113 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.827083    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" podUID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Nov 03 21:00:45.989111 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.987726    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:45.989111 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:45.988959    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:46.011529 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:46.006100    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:00:46.418565 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:46.418502    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:46.419257 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:46.419239    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:46.424229 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:46.421979    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:00:47.038346 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:47.038316    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:47.046475 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:47.046441    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:47.049826 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:47.049801    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:00:47.194983 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:47.194940    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:47.195348 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:47.195321    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:47.196359 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:47.196337    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:00:57.841744 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:57.841697    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:57.903187 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:57.903038    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:57.974580 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:57.972870    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:58.151466 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:58.151367    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:58.197751 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:58.197703    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 03 21:00:58.442705 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:58.437746    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:58.910308 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:58.910266    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:58.911534 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:58.911496    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 21:00:58.981654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:58.980717    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:58.981654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:58.981120    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 21:00:59.374685 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:59.372676    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 21:00:59.376611 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:59.376574    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 21:00:59.452568 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:59.443051    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:00:59.452568 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:59.443706    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 21:00:59.587984 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:59.587956    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 21:00:59.767308 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:00:59.767181    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 21:01:01.690418 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:01:01.690337    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:01:21.376831 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.376791    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"]
Nov 03 21:01:21.380887 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.380864    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.410939 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.410913    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkwmm\" (UniqueName: \"kubernetes.io/projected/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-kube-api-access-dkwmm\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.411074 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.410951    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.411074 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.410969    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.411201 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.411130    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.411259 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.411236    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.411301 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.411280    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.411347 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.411299    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.411419 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.411401    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.411533 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.411507    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.501032 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.500993    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"]
Nov 03 21:01:21.513821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.512607    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.513821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.512741    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkwmm\" (UniqueName: \"kubernetes.io/projected/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-kube-api-access-dkwmm\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.513821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.512784    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.513821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.512808    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.513821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.512904    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.513821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.512989    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.513821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.513012    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.513821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.513038    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.513821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.513138    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.513821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.513323    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-credential-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.513821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.513512    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-workload-certs\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.513821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.513606    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-workload-socket\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.513821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.513767    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-data\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.514610 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.513832    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istiod-ca-cert\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.516021 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.515994    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-envoy\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.516168 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.516118    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-podinfo\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.575444 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.575412    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-token\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.575596 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.575553    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkwmm\" (UniqueName: \"kubernetes.io/projected/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-kube-api-access-dkwmm\") pod \"e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") " pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:21.696563 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:21.696468    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:22.138747 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.138717    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"]
Nov 03 21:01:22.139229 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:01:22.139201    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod122dd6bc_5f41_4ad1_bd2c_568c4a79baa7.slice/crio-ca78865d9b9c51fc2a0423962d7446549de0d3c2234fe6c60c187935de19cc65 WatchSource:0}: Error finding container ca78865d9b9c51fc2a0423962d7446549de0d3c2234fe6c60c187935de19cc65: Status 404 returned error can't find the container with id ca78865d9b9c51fc2a0423962d7446549de0d3c2234fe6c60c187935de19cc65
Nov 03 21:01:22.141707 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.141671    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:22.141802 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.141753    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:22.141863 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.141831    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:22.189138 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.189105    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"]
Nov 03 21:01:22.195666 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.195623    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.222369 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.222341    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.222473 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.222401    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.222473 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.222454    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptfrr\" (UniqueName: \"kubernetes.io/projected/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-kube-api-access-ptfrr\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.222555 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.222478    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.222555 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.222516    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.222621 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.222582    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.222709 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.222694    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.222771 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.222758    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.222807 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.222789    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.309652 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.309277    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"]
Nov 03 21:01:22.324567 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.324513    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.324779 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.324579    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.324779 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.324666    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.324779 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.324691    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.324779 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.324733    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptfrr\" (UniqueName: \"kubernetes.io/projected/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-kube-api-access-ptfrr\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.324779 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.324769    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.325078 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.324826    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.325078 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.324915    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.325078 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.325022    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.325862 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.325488    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.326225 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.326194    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.326842 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.326479    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.326842 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.326785    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.327001 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.326876    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.329807 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.329778    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.329909 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.329864    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.394312 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.393767    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptfrr\" (UniqueName: \"kubernetes.io/projected/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-kube-api-access-ptfrr\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.394312 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.394220    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.511940 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.511910    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:22.988650 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.986611    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"]
Nov 03 21:01:22.998310 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.997186    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:22.998310 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.997275    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:22.998310 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:22.997315    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:23.104882 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:23.104846    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj" event={"ID":"260b8942-a2ee-41fa-b64d-daa30ecc0bc1","Type":"ContainerStarted","Data":"418e1f90c4c5b3c794bd3e2ce213aa7b0a81b74467d866337a8b3f057a51f691"}
Nov 03 21:01:23.106834 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:23.106804    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r" event={"ID":"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7","Type":"ContainerStarted","Data":"ca79ca51f105055588eff90897589d93d24eb0e699ee4936ce7258e15cca61b6"}
Nov 03 21:01:23.107175 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:23.106861    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r" event={"ID":"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7","Type":"ContainerStarted","Data":"ca78865d9b9c51fc2a0423962d7446549de0d3c2234fe6c60c187935de19cc65"}
Nov 03 21:01:23.230703 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:23.230646    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r" podStartSLOduration=2.230612291 podStartE2EDuration="2.230612291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:01:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:01:23.227659253 +0000 UTC m=+1285.772365617" watchObservedRunningTime="2026-11-03 21:01:23.230612291 +0000 UTC m=+1285.775318660"
Nov 03 21:01:23.697873 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:23.697836    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:23.701196 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:23.701164    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 21:01:23.701337 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:23.701229    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r" podUID="122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 21:01:24.135583 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:24.135534    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj" event={"ID":"260b8942-a2ee-41fa-b64d-daa30ecc0bc1","Type":"ContainerStarted","Data":"59e81033fc9b04024f9a73d70a58fbfdd43d1365477b90d4389e88975f4f57a3"}
Nov 03 21:01:24.513651 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:24.513498    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:24.513651 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:24.513619    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 21:01:24.513908 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:24.513688    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj" podUID="260b8942-a2ee-41fa-b64d-daa30ecc0bc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 21:01:24.703654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:24.703073    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 21:01:24.703654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:24.703134    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r" podUID="122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 21:01:25.513656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:25.513050    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 21:01:25.513656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:25.513113    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj" podUID="260b8942-a2ee-41fa-b64d-daa30ecc0bc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 21:01:25.697185 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:25.697146    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 21:01:25.697444 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:25.697417    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r" podUID="122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 21:01:26.537476 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:26.537435    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 21:01:26.538726 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:26.537711    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj" podUID="260b8942-a2ee-41fa-b64d-daa30ecc0bc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 21:01:26.704673 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:26.704612    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 21:01:26.704989 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:26.704941    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r" podUID="122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 21:01:27.529376 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:27.529331    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 21:01:27.529596 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:27.529400    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj" podUID="260b8942-a2ee-41fa-b64d-daa30ecc0bc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 21:01:27.697046 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:27.697003    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 21:01:27.697649 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:27.697595    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r" podUID="122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 21:01:28.520241 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:28.520198    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 21:01:28.529479 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:28.529433    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj" podUID="260b8942-a2ee-41fa-b64d-daa30ecc0bc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 21:01:28.697054 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:28.697015    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 21:01:28.697730 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:28.697694    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r" podUID="122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 21:01:29.533889 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:29.533852    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Nov 03 21:01:29.534160 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:29.534123    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj" podUID="260b8942-a2ee-41fa-b64d-daa30ecc0bc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Nov 03 21:01:30.208893 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:30.208841    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:30.211314 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:30.211293    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:30.214988 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:30.214953    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:01:30.382987 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:30.382926    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj" podStartSLOduration=8.382908398 podStartE2EDuration="8.382908398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:01:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:01:24.292981372 +0000 UTC m=+1286.837687744" watchObservedRunningTime="2026-11-03 21:01:30.382908398 +0000 UTC m=+1292.927614772"
Nov 03 21:01:31.007151 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:31.007118    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:31.007808 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:31.007748    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:31.009744 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:31.009724    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:01:31.705273 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:01:31.705232    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:01:33.377687 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:33.376787    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 21:01:33.377687 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:33.377471    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:33.625921 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:33.625876    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:33.650495 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:33.626176    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:33.659654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:33.626209    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Nov 03 21:01:33.672786 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:33.672736    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:33.793665 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:33.777502    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:33.803687 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:33.789651    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:33.859723 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:33.855730    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:33.859723 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:33.858572    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:34.041786 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.041735    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:34.173786 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.173733    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:34.202871 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.202826    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 03 21:01:34.431718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.407893    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:34.431718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.408295    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 21:01:34.431718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.408958    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 21:01:34.431718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.409017    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 21:01:34.674656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.602178    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:34.678655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.675002    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:34.708713 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.683118    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 21:01:34.732860 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.709928    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:34.733452 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.713862    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Nov 03 21:01:34.792653 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.713893    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": context deadline exceeded"
Nov 03 21:01:34.863214 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.734106    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 21:01:34.863965 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.863944    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 21:01:34.864198 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.864181    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 21:01:34.865356 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.800754    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": context deadline exceeded"
Nov 03 21:01:34.894090 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.863822    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Nov 03 21:01:34.894692 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.894665    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 21:01:34.894930 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.894915    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 21:01:34.908663 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.801353    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:34.912660 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.863779    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 03 21:01:34.960738 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.917366    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:01:34.960738 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:34.920175    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 21:01:35.051656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.043739    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:35.051656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.044185    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 21:01:35.200695 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.197207    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:35.203754 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.203706    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:35.204300 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.204275    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 21:01:35.204482 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.204467    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:01:35.287546 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.287414    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/health\": context deadline exceeded"
Nov 03 21:01:35.415046 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.415002    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:35.438529 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.438204    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:35.499746 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.499262    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 21:01:35.513128 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.513099    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 21:01:35.576196 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.576107    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 21:01:35.721812 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.721533    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:01:35.753650 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.751010    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 21:01:35.753650 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.751080    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 21:01:35.811361 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.811329    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 21:01:35.833983 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.833895    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 21:01:35.899750 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.899705    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:35.923653 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.904248    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:35.961119 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.961091    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 21:01:35.989783 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:35.989011    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 21:01:36.050395 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:36.037061    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 21:01:36.050395 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:36.046746    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:01:36.084858 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:36.084779    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 21:01:36.089301 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:36.089271    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:01:37.307916 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.307880    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"]
Nov 03 21:01:37.313513 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.313480    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.366237 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.366197    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.366464 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.366353    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.366464 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.366397    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.366464 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.366424    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.366599 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.366480    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.366599 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.366530    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.366724 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.366603    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.366724 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.366668    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27r2c\" (UniqueName: \"kubernetes.io/projected/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-kube-api-access-27r2c\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.366793 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.366723    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.384096 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.384057    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"]
Nov 03 21:01:37.476368 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.476337    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.476553 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.476404    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.476553 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.476430    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.476553 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.476486    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.476553 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.476533    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.476988 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.476603    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.476988 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.476674    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27r2c\" (UniqueName: \"kubernetes.io/projected/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-kube-api-access-27r2c\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.476988 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.476731    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.476988 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.476816    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.477302 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.477280    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-workload-certs\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.477512 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.477492    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-credential-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.478406 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.478378    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istiod-ca-cert\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.481402 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.481379    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-data\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.481853 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.481833    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-workload-socket\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.482010 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.481983    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-podinfo\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.482155 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.482130    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-envoy\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.548548 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.548519    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-token\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.550932 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.550898    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27r2c\" (UniqueName: \"kubernetes.io/projected/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-kube-api-access-27r2c\") pod \"e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") " pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:37.629144 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:37.629060    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:38.215170 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:01:38.215136    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e886cd0_4ff5_4364_8a6b_3124b16e9cdc.slice/crio-437ba0caacdd97d2f9f2d521f8041ca3bb1884b64262b9e01a3c06cc5019249b WatchSource:0}: Error finding container 437ba0caacdd97d2f9f2d521f8041ca3bb1884b64262b9e01a3c06cc5019249b: Status 404 returned error can't find the container with id 437ba0caacdd97d2f9f2d521f8041ca3bb1884b64262b9e01a3c06cc5019249b
Nov 03 21:01:38.215326 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:38.215203    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"]
Nov 03 21:01:38.217969 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:38.217937    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:38.218045 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:38.218004    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:38.218045 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:38.218030    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:01:38.514065 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:38.510523    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46" event={"ID":"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc","Type":"ContainerStarted","Data":"adb77385ab75a80eda9faa7d63f659dae808b1833f9717052050e00f4645c03c"}
Nov 03 21:01:38.514065 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:38.510590    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46" event={"ID":"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc","Type":"ContainerStarted","Data":"437ba0caacdd97d2f9f2d521f8041ca3bb1884b64262b9e01a3c06cc5019249b"}
Nov 03 21:01:38.629575 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:38.629544    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:38.636559 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:38.636497    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46" podStartSLOduration=1.636476466 podStartE2EDuration="1.636476466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:01:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:01:38.63442179 +0000 UTC m=+1301.179128162" watchObservedRunningTime="2026-11-03 21:01:38.636476466 +0000 UTC m=+1301.181182840"
Nov 03 21:01:39.631170 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:39.631137    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 03 21:01:39.631571 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:39.631187    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46" podUID="5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 03 21:01:40.629781 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:40.629744    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 03 21:01:40.629976 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:40.629799    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46" podUID="5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 03 21:01:41.629904 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:41.629866    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 03 21:01:41.630337 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:41.629938    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46" podUID="5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 03 21:01:42.630221 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:42.630181    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 03 21:01:42.630613 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:42.630251    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46" podUID="5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 03 21:01:43.629696 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:43.629655    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 03 21:01:43.629859 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:43.629708    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46" podUID="5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 03 21:01:44.630381 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:44.630337    4993 patch_prober.go:29] interesting pod/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Nov 03 21:01:44.630890 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:44.630424    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46" podUID="5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Nov 03 21:01:45.766391 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:45.766359    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:45.766952 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:45.766912    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:45.768234 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:45.768211    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:01:52.206741 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:52.206704    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"]
Nov 03 21:01:52.209441 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:52.206930    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x" podUID="98f5c9db-d68a-4f60-8bde-d8a25acd1275" containerName="istio-proxy" containerID="cri-o://6cbe7a39468bf3dfc11e430b71641613d359f5b70cc4d72e73e3efa2fe2e3f75" gracePeriod=30
Nov 03 21:01:52.256693 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:52.256653    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"]
Nov 03 21:01:52.256935 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:52.256895    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs" podUID="c3861b24-aee3-4389-b01e-325fa453a7ce" containerName="istio-proxy" containerID="cri-o://b1f78712c6f40d69424c787e8bb7787c3900b2ac86d656e6260db959a27736a0" gracePeriod=30
Nov 03 21:01:57.541296 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.541260    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 21:01:57.575600 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.575570    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 21:01:57.657085 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.657047    4993 generic.go:356] "Generic (PLEG): container finished" podID="98f5c9db-d68a-4f60-8bde-d8a25acd1275" containerID="6cbe7a39468bf3dfc11e430b71641613d359f5b70cc4d72e73e3efa2fe2e3f75" exitCode=0
Nov 03 21:01:57.657282 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.657115    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"
Nov 03 21:01:57.657282 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.657151    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x" event={"ID":"98f5c9db-d68a-4f60-8bde-d8a25acd1275","Type":"ContainerDied","Data":"6cbe7a39468bf3dfc11e430b71641613d359f5b70cc4d72e73e3efa2fe2e3f75"}
Nov 03 21:01:57.657282 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.657222    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x" event={"ID":"98f5c9db-d68a-4f60-8bde-d8a25acd1275","Type":"ContainerDied","Data":"274f64d2a8d99f0552cf447b88a4e63c69fda716e182c38e07fcc109ac09b1ea"}
Nov 03 21:01:57.657282 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.657248    4993 scope.go:122] "RemoveContainer" containerID="6cbe7a39468bf3dfc11e430b71641613d359f5b70cc4d72e73e3efa2fe2e3f75"
Nov 03 21:01:57.659086 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.659062    4993 generic.go:356] "Generic (PLEG): container finished" podID="c3861b24-aee3-4389-b01e-325fa453a7ce" containerID="b1f78712c6f40d69424c787e8bb7787c3900b2ac86d656e6260db959a27736a0" exitCode=0
Nov 03 21:01:57.659220 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.659202    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs" event={"ID":"c3861b24-aee3-4389-b01e-325fa453a7ce","Type":"ContainerDied","Data":"b1f78712c6f40d69424c787e8bb7787c3900b2ac86d656e6260db959a27736a0"}
Nov 03 21:01:57.659294 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.659247    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs" event={"ID":"c3861b24-aee3-4389-b01e-325fa453a7ce","Type":"ContainerDied","Data":"f0eb6f6ee258873ace71e96dbc36d145b654bd5a52ef52a38ac7b1b28da018a5"}
Nov 03 21:01:57.659353 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.659341    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"
Nov 03 21:01:57.670593 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.670568    4993 scope.go:122] "RemoveContainer" containerID="6cbe7a39468bf3dfc11e430b71641613d359f5b70cc4d72e73e3efa2fe2e3f75"
Nov 03 21:01:57.670925 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:01:57.670902    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cbe7a39468bf3dfc11e430b71641613d359f5b70cc4d72e73e3efa2fe2e3f75\": container with ID starting with 6cbe7a39468bf3dfc11e430b71641613d359f5b70cc4d72e73e3efa2fe2e3f75 not found: ID does not exist" containerID="6cbe7a39468bf3dfc11e430b71641613d359f5b70cc4d72e73e3efa2fe2e3f75"
Nov 03 21:01:57.671013 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.670936    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cbe7a39468bf3dfc11e430b71641613d359f5b70cc4d72e73e3efa2fe2e3f75"} err="failed to get container status \"6cbe7a39468bf3dfc11e430b71641613d359f5b70cc4d72e73e3efa2fe2e3f75\": rpc error: code = NotFound desc = could not find container \"6cbe7a39468bf3dfc11e430b71641613d359f5b70cc4d72e73e3efa2fe2e3f75\": container with ID starting with 6cbe7a39468bf3dfc11e430b71641613d359f5b70cc4d72e73e3efa2fe2e3f75 not found: ID does not exist"
Nov 03 21:01:57.671013 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.670959    4993 scope.go:122] "RemoveContainer" containerID="b1f78712c6f40d69424c787e8bb7787c3900b2ac86d656e6260db959a27736a0"
Nov 03 21:01:57.680738 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.680721    4993 scope.go:122] "RemoveContainer" containerID="b1f78712c6f40d69424c787e8bb7787c3900b2ac86d656e6260db959a27736a0"
Nov 03 21:01:57.681022 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:01:57.681000    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1f78712c6f40d69424c787e8bb7787c3900b2ac86d656e6260db959a27736a0\": container with ID starting with b1f78712c6f40d69424c787e8bb7787c3900b2ac86d656e6260db959a27736a0 not found: ID does not exist" containerID="b1f78712c6f40d69424c787e8bb7787c3900b2ac86d656e6260db959a27736a0"
Nov 03 21:01:57.681109 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.681028    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1f78712c6f40d69424c787e8bb7787c3900b2ac86d656e6260db959a27736a0"} err="failed to get container status \"b1f78712c6f40d69424c787e8bb7787c3900b2ac86d656e6260db959a27736a0\": rpc error: code = NotFound desc = could not find container \"b1f78712c6f40d69424c787e8bb7787c3900b2ac86d656e6260db959a27736a0\": container with ID starting with b1f78712c6f40d69424c787e8bb7787c3900b2ac86d656e6260db959a27736a0 not found: ID does not exist"
Nov 03 21:01:57.691719 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.691698    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3861b24-aee3-4389-b01e-325fa453a7ce-kube-api-access-n9l7b\" (UniqueName: \"kubernetes.io/projected/c3861b24-aee3-4389-b01e-325fa453a7ce-kube-api-access-n9l7b\") pod \"c3861b24-aee3-4389-b01e-325fa453a7ce\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") "
Nov 03 21:01:57.691804 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.691750    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-data\") pod \"c3861b24-aee3-4389-b01e-325fa453a7ce\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") "
Nov 03 21:01:57.691899 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.691860    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-podinfo\") pod \"c3861b24-aee3-4389-b01e-325fa453a7ce\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") "
Nov 03 21:01:57.691957 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.691941    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-envoy\") pod \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") "
Nov 03 21:01:57.692015 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.691971    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-data\") pod \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") "
Nov 03 21:01:57.692015 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.691974    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-data" pod "c3861b24-aee3-4389-b01e-325fa453a7ce" (UID: "c3861b24-aee3-4389-b01e-325fa453a7ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.692015 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.692002    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-podinfo\") pod \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") "
Nov 03 21:01:57.692180 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.692077    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-credential-socket\") pod \"c3861b24-aee3-4389-b01e-325fa453a7ce\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") "
Nov 03 21:01:57.692180 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.692125    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istiod-ca-cert\") pod \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") "
Nov 03 21:01:57.692287 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.692202    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-workload-socket\") pod \"c3861b24-aee3-4389-b01e-325fa453a7ce\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") "
Nov 03 21:01:57.692287 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.692239    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-workload-socket\") pod \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") "
Nov 03 21:01:57.692287 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.692281    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-workload-certs\") pod \"c3861b24-aee3-4389-b01e-325fa453a7ce\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") "
Nov 03 21:01:57.692447 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.692312    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-data" pod "98f5c9db-d68a-4f60-8bde-d8a25acd1275" (UID: "98f5c9db-d68a-4f60-8bde-d8a25acd1275"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.692447 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.692337    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-token\" (UniqueName: \"kubernetes.io/projected/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-token\") pod \"c3861b24-aee3-4389-b01e-325fa453a7ce\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") "
Nov 03 21:01:57.692447 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.692375    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-token\" (UniqueName: \"kubernetes.io/projected/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-token\") pod \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") "
Nov 03 21:01:57.692608 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.692574    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-workload-socket" pod "c3861b24-aee3-4389-b01e-325fa453a7ce" (UID: "c3861b24-aee3-4389-b01e-325fa453a7ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.692803 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.692756    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-credential-socket" pod "c3861b24-aee3-4389-b01e-325fa453a7ce" (UID: "c3861b24-aee3-4389-b01e-325fa453a7ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.692892 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.692817    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-workload-certs" pod "c3861b24-aee3-4389-b01e-325fa453a7ce" (UID: "c3861b24-aee3-4389-b01e-325fa453a7ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.693014 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.692923    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istiod-ca-cert" pod "98f5c9db-d68a-4f60-8bde-d8a25acd1275" (UID: "98f5c9db-d68a-4f60-8bde-d8a25acd1275"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:01:57.693097 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.693011    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-workload-socket" pod "98f5c9db-d68a-4f60-8bde-d8a25acd1275" (UID: "98f5c9db-d68a-4f60-8bde-d8a25acd1275"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.693192 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.693172    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98f5c9db-d68a-4f60-8bde-d8a25acd1275-kube-api-access-n92d4\" (UniqueName: \"kubernetes.io/projected/98f5c9db-d68a-4f60-8bde-d8a25acd1275-kube-api-access-n92d4\") pod \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") "
Nov 03 21:01:57.693499 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.693477    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-credential-socket\") pod \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") "
Nov 03 21:01:57.693802 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.693523    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-envoy\") pod \"c3861b24-aee3-4389-b01e-325fa453a7ce\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") "
Nov 03 21:01:57.693802 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.693594    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-workload-certs\") pod \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\" (UID: \"98f5c9db-d68a-4f60-8bde-d8a25acd1275\") "
Nov 03 21:01:57.693802 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.693672    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3861b24-aee3-4389-b01e-325fa453a7ce-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3861b24-aee3-4389-b01e-325fa453a7ce-istiod-ca-cert\") pod \"c3861b24-aee3-4389-b01e-325fa453a7ce\" (UID: \"c3861b24-aee3-4389-b01e-325fa453a7ce\") "
Nov 03 21:01:57.694391 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.694337    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.694391 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.694366    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.694391 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.694379    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.694391 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.694392    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.694737 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.694424    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.694737 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.694439    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.694737 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.694453    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.694942 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.694786    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-podinfo" pod "c3861b24-aee3-4389-b01e-325fa453a7ce" (UID: "c3861b24-aee3-4389-b01e-325fa453a7ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:01:57.694942 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.694908    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-credential-socket" pod "98f5c9db-d68a-4f60-8bde-d8a25acd1275" (UID: "98f5c9db-d68a-4f60-8bde-d8a25acd1275"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.695049 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.694983    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3861b24-aee3-4389-b01e-325fa453a7ce-kube-api-access-n9l7b" pod "c3861b24-aee3-4389-b01e-325fa453a7ce" (UID: "c3861b24-aee3-4389-b01e-325fa453a7ce"). InnerVolumeSpecName "kube-api-access-n9l7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:01:57.695103 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.695052    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3861b24-aee3-4389-b01e-325fa453a7ce-istiod-ca-cert" pod "c3861b24-aee3-4389-b01e-325fa453a7ce" (UID: "c3861b24-aee3-4389-b01e-325fa453a7ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:01:57.695238 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.695169    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-podinfo" pod "98f5c9db-d68a-4f60-8bde-d8a25acd1275" (UID: "98f5c9db-d68a-4f60-8bde-d8a25acd1275"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:01:57.695238 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.695251    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-envoy" pod "98f5c9db-d68a-4f60-8bde-d8a25acd1275" (UID: "98f5c9db-d68a-4f60-8bde-d8a25acd1275"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.695615 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.695482    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-workload-certs" pod "98f5c9db-d68a-4f60-8bde-d8a25acd1275" (UID: "98f5c9db-d68a-4f60-8bde-d8a25acd1275"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.695730 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.695669    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-token" pod "98f5c9db-d68a-4f60-8bde-d8a25acd1275" (UID: "98f5c9db-d68a-4f60-8bde-d8a25acd1275"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:01:57.696582 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.696565    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-envoy" pod "c3861b24-aee3-4389-b01e-325fa453a7ce" (UID: "c3861b24-aee3-4389-b01e-325fa453a7ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:01:57.696709 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.696692    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-token" pod "c3861b24-aee3-4389-b01e-325fa453a7ce" (UID: "c3861b24-aee3-4389-b01e-325fa453a7ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:01:57.696854 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.696833    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98f5c9db-d68a-4f60-8bde-d8a25acd1275-kube-api-access-n92d4" pod "98f5c9db-d68a-4f60-8bde-d8a25acd1275" (UID: "98f5c9db-d68a-4f60-8bde-d8a25acd1275"). InnerVolumeSpecName "kube-api-access-n92d4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:01:57.796174 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.796077    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.796174 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.796105    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.796174 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.796115    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n92d4\" (UniqueName: \"kubernetes.io/projected/98f5c9db-d68a-4f60-8bde-d8a25acd1275-kube-api-access-n92d4\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.796174 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.796125    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.796174 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.796134    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.796174 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.796142    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.796174 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.796151    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3861b24-aee3-4389-b01e-325fa453a7ce-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.796174 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.796160    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9l7b\" (UniqueName: \"kubernetes.io/projected/c3861b24-aee3-4389-b01e-325fa453a7ce-kube-api-access-n9l7b\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.796174 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.796168    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3861b24-aee3-4389-b01e-325fa453a7ce-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.796174 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.796180    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:57.796859 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:57.796190    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98f5c9db-d68a-4f60-8bde-d8a25acd1275-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:01:58.088243 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:58.088151    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"]
Nov 03 21:01:58.098599 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:58.098565    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-zcz4x"]
Nov 03 21:01:58.211426 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:58.211392    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"]
Nov 03 21:01:58.215779 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:58.215732    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-slbzs"]
Nov 03 21:01:59.579022 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:59.578984    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98f5c9db-d68a-4f60-8bde-d8a25acd1275" path="/var/lib/kubelet/pods/98f5c9db-d68a-4f60-8bde-d8a25acd1275/volumes"
Nov 03 21:01:59.579517 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:01:59.579473    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3861b24-aee3-4389-b01e-325fa453a7ce" path="/var/lib/kubelet/pods/c3861b24-aee3-4389-b01e-325fa453a7ce/volumes"
Nov 03 21:02:01.673836 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:02:01.673788    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:02:08.597770 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:08.597721    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:08.668780 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:08.668729    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Nov 03 21:02:08.813702 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:08.810170    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Nov 03 21:02:08.862837 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:08.862731    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:08.907503 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:08.907442    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": context deadline exceeded"
Nov 03 21:02:08.996657 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:08.995868    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.046652 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.045767    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.096794 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.096741    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.141895 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.141790    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.235120 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.226091    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.434158 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.434047    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.514152 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.514101    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 03 21:02:09.599819 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.599751    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.601204 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.601181    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:02:09.772653 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.769714    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Nov 03 21:02:09.772653 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.771373    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 21:02:09.873752 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.873692    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.896723 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.896676    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 21:02:09.915584 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.915207    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.928657 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.918718    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:09.928657 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.918846    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 21:02:09.928657 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:09.926795    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 21:02:10.001729 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.001313    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:10.001935 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.001835    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 21:02:10.205823 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.205712    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:10.391652 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.220725    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 03 21:02:10.391652 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.238488    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:10.391652 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.378740    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 21:02:10.391652 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.378820    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 21:02:10.391652 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.378861    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:02:10.485655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.485353    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:10.549126 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.549083    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:10.549702 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.549673    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": context deadline exceeded"
Nov 03 21:02:10.553107 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.551511    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 21:02:10.609912 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.609859    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:10.658559 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.658527    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 21:02:10.675502 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.675471    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 21:02:10.786059 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.781376    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Nov 03 21:02:10.940839 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.940788    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": context deadline exceeded"
Nov 03 21:02:10.941751 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.941710    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:10.943807 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:10.943778    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 03 21:02:11.049301 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.027157    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.315884 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.304073    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.412341 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.412296    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.412952 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.412917    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 03 21:02:11.648938 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.648736    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.650416 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.650235    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.655920 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.655431    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.696954 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.675891    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.696954 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.678308    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.801825 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.787240    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.822655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.822456    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 21:02:11.822655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.822522    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 21:02:11.840721 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.840427    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 21:02:11.850658 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.841347    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 21:02:11.905707 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.903019    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/health\": context deadline exceeded"
Nov 03 21:02:11.989655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.958947    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.989655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.960712    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:11.989655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.965110    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": context deadline exceeded"
Nov 03 21:02:11.989655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:11.989427    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 21:02:12.100653 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:12.036393    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:12.108287 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:12.089398    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:12.158899 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:12.158813    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 21:02:12.223868 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:12.223826    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": context deadline exceeded"
Nov 03 21:02:12.232562 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:12.232402    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 21:02:12.234993 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:12.234974    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 21:02:12.326652 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:12.325875    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:02:12.369780 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:12.369752    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 21:02:12.413841 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:12.413734    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:12.444120 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:12.444087    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:02:12.445644 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:12.445605    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 21:02:19.055861 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:19.055825    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7364-2ff420"]
Nov 03 21:02:19.056641 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:19.056605    4993 cpu_manager.go:414] "containerMap: removing container" podUID="c3861b24-aee3-4389-b01e-325fa453a7ce" containerName="istio-proxy"
Nov 03 21:02:19.056709 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:19.056651    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3861b24-aee3-4389-b01e-325fa453a7ce" containerName="istio-proxy"
Nov 03 21:02:19.056709 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:19.056687    4993 cpu_manager.go:414] "containerMap: removing container" podUID="98f5c9db-d68a-4f60-8bde-d8a25acd1275" containerName="istio-proxy"
Nov 03 21:02:19.056709 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:19.056694    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98f5c9db-d68a-4f60-8bde-d8a25acd1275" containerName="istio-proxy"
Nov 03 21:02:19.056913 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:19.056893    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="98f5c9db-d68a-4f60-8bde-d8a25acd1275" containerName="istio-proxy"
Nov 03 21:02:19.056962 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:19.056929    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3861b24-aee3-4389-b01e-325fa453a7ce" containerName="istio-proxy"
Nov 03 21:02:19.060994 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:19.060970    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-2ff420"
Nov 03 21:02:19.115004 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:19.114965    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-2ff420"]
Nov 03 21:02:19.161046 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:19.160998    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-nrr89\""
Nov 03 21:02:19.235167 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:19.235124    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfngm\" (UniqueName: \"kubernetes.io/projected/66b1d4fe-410d-47e0-a1a0-c13a71a8d938-kube-api-access-xfngm\") pod \"test-curl-7364-2ff420\" (UID: \"66b1d4fe-410d-47e0-a1a0-c13a71a8d938\") " pod="opendatahub/test-curl-7364-2ff420"
Nov 03 21:02:19.336246 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:19.336159    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfngm\" (UniqueName: \"kubernetes.io/projected/66b1d4fe-410d-47e0-a1a0-c13a71a8d938-kube-api-access-xfngm\") pod \"test-curl-7364-2ff420\" (UID: \"66b1d4fe-410d-47e0-a1a0-c13a71a8d938\") " pod="opendatahub/test-curl-7364-2ff420"
Nov 03 21:02:19.397968 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:19.397929    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfngm\" (UniqueName: \"kubernetes.io/projected/66b1d4fe-410d-47e0-a1a0-c13a71a8d938-kube-api-access-xfngm\") pod \"test-curl-7364-2ff420\" (UID: \"66b1d4fe-410d-47e0-a1a0-c13a71a8d938\") " pod="opendatahub/test-curl-7364-2ff420"
Nov 03 21:02:19.673999 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:19.673896    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-2ff420"
Nov 03 21:02:20.060180 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:20.060154    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-2ff420"]
Nov 03 21:02:20.061705 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:02:20.061673    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66b1d4fe_410d_47e0_a1a0_c13a71a8d938.slice/crio-341bc91432504b16a6d49c78f8295a8fbcae7fb64a0a564acad39254df55dc68 WatchSource:0}: Error finding container 341bc91432504b16a6d49c78f8295a8fbcae7fb64a0a564acad39254df55dc68: Status 404 returned error can't find the container with id 341bc91432504b16a6d49c78f8295a8fbcae7fb64a0a564acad39254df55dc68
Nov 03 21:02:20.064239 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:20.064218    4993 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 21:02:20.819554 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:20.819504    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-2ff420" event={"ID":"66b1d4fe-410d-47e0-a1a0-c13a71a8d938","Type":"ContainerStarted","Data":"341bc91432504b16a6d49c78f8295a8fbcae7fb64a0a564acad39254df55dc68"}
Nov 03 21:02:21.582404 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.582366    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"]
Nov 03 21:02:21.587906 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.587882    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.668180 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.668137    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82ef457f-b231-432d-9c89-52e22180cea2-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.668506 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.668187    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82ef457f-b231-432d-9c89-52e22180cea2-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.668506 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.668289    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mpck\" (UniqueName: \"kubernetes.io/projected/82ef457f-b231-432d-9c89-52e22180cea2-kube-api-access-4mpck\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.668506 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.668323    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.668506 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.668406    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.668748 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.668541    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82ef457f-b231-432d-9c89-52e22180cea2-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.668748 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.668599    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.668748 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.668669    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.668748 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.668709    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.732513 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.732475    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"]
Nov 03 21:02:21.770435 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.770401    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82ef457f-b231-432d-9c89-52e22180cea2-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.770698 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.770447    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82ef457f-b231-432d-9c89-52e22180cea2-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.770698 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.770484    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mpck\" (UniqueName: \"kubernetes.io/projected/82ef457f-b231-432d-9c89-52e22180cea2-kube-api-access-4mpck\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.770847 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.770698    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.770847 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.770783    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.770958 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.770919    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82ef457f-b231-432d-9c89-52e22180cea2-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.771023 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.770998    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.771082 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.771041    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.771133 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.771105    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.771133 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.771114    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-workload-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.771401 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.771381    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-credential-socket\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.771747 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.771722    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-workload-certs\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.771747 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.771735    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82ef457f-b231-432d-9c89-52e22180cea2-istiod-ca-cert\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.771900 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.771733    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-istio-data\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.773234 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.773215    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-istio-envoy\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.773993 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.773967    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82ef457f-b231-432d-9c89-52e22180cea2-istio-podinfo\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.830141 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.830094    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-2ff420" event={"ID":"66b1d4fe-410d-47e0-a1a0-c13a71a8d938","Type":"ContainerStarted","Data":"cd946b7177645a2624ffd908f90c1ac88458c9667d07d67363fbf7d6c7d024b5"}
Nov 03 21:02:21.834220 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.834147    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mpck\" (UniqueName: \"kubernetes.io/projected/82ef457f-b231-432d-9c89-52e22180cea2-kube-api-access-4mpck\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.835344 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.835315    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82ef457f-b231-432d-9c89-52e22180cea2-istio-token\") pod \"e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") " pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.902855 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.902823    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:21.949030 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:21.948945    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7364-2ff420" podStartSLOduration=2.135115286 podStartE2EDuration="2.948928337s" totalImagesPullingTime="813.813051ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:02:21.947784325 +0000 UTC m=+1344.492490700" watchObservedRunningTime="2026-11-03 21:02:21.948928337 +0000 UTC m=+1344.493634708"
Nov 03 21:02:22.338182 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:22.338146    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"]
Nov 03 21:02:22.338325 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:02:22.338258    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ef457f_b231_432d_9c89_52e22180cea2.slice/crio-698dde6abaf066c504f6dc0844ac962e77fbed0bcbee67c9beb51e052b3a06f7 WatchSource:0}: Error finding container 698dde6abaf066c504f6dc0844ac962e77fbed0bcbee67c9beb51e052b3a06f7: Status 404 returned error can't find the container with id 698dde6abaf066c504f6dc0844ac962e77fbed0bcbee67c9beb51e052b3a06f7
Nov 03 21:02:22.341180 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:22.341145    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:02:22.341292 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:22.341231    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:02:22.341292 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:22.341280    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:02:22.837891 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:22.837852    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr" event={"ID":"82ef457f-b231-432d-9c89-52e22180cea2","Type":"ContainerStarted","Data":"ea0bc4938dc734c4343b185809cb4d48f2081f67e46034d706b67b03b9fa6026"}
Nov 03 21:02:22.838337 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:22.837922    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr" event={"ID":"82ef457f-b231-432d-9c89-52e22180cea2","Type":"ContainerStarted","Data":"698dde6abaf066c504f6dc0844ac962e77fbed0bcbee67c9beb51e052b3a06f7"}
Nov 03 21:02:22.908657 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:22.906744    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:22.960373 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:22.960317    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr" podStartSLOduration=1.960300793 podStartE2EDuration="1.960300793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:02:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:02:22.957812252 +0000 UTC m=+1345.502518623" watchObservedRunningTime="2026-11-03 21:02:22.960300793 +0000 UTC m=+1345.505007234"
Nov 03 21:02:23.620901 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:23.620868    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7364-2ff420"]
Nov 03 21:02:23.627236 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:23.627202    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7364-2ff420"]
Nov 03 21:02:23.849057 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:23.848388    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7364-2ff420" podUID="66b1d4fe-410d-47e0-a1a0-c13a71a8d938" containerName="test-curl-7364-2ff420" containerID="cri-o://cd946b7177645a2624ffd908f90c1ac88458c9667d07d67363fbf7d6c7d024b5" gracePeriod=2
Nov 03 21:02:23.901771 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:02:23.901680    4993 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7364-2ff420\" is forbidden: User \"system:node:ip-10-0-142-223.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-223.us-west-1.compute.internal' and this object" podUID="66b1d4fe-410d-47e0-a1a0-c13a71a8d938" pod="opendatahub/test-curl-7364-2ff420"
Nov 03 21:02:23.905839 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:23.905809    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 21:02:23.906026 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:23.905857    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr" podUID="82ef457f-b231-432d-9c89-52e22180cea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 21:02:24.909948 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:24.909848    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 21:02:24.909948 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:24.909915    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr" podUID="82ef457f-b231-432d-9c89-52e22180cea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 21:02:25.904984 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:25.904809    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 21:02:25.904984 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:25.904883    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr" podUID="82ef457f-b231-432d-9c89-52e22180cea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 21:02:26.150224 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:26.150192    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-2ff420"
Nov 03 21:02:26.188700 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:26.178300    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66b1d4fe-410d-47e0-a1a0-c13a71a8d938-kube-api-access-xfngm\" (UniqueName: \"kubernetes.io/projected/66b1d4fe-410d-47e0-a1a0-c13a71a8d938-kube-api-access-xfngm\") pod \"66b1d4fe-410d-47e0-a1a0-c13a71a8d938\" (UID: \"66b1d4fe-410d-47e0-a1a0-c13a71a8d938\") "
Nov 03 21:02:26.193815 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:26.193782    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66b1d4fe-410d-47e0-a1a0-c13a71a8d938-kube-api-access-xfngm" pod "66b1d4fe-410d-47e0-a1a0-c13a71a8d938" (UID: "66b1d4fe-410d-47e0-a1a0-c13a71a8d938"). InnerVolumeSpecName "kube-api-access-xfngm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:02:26.285866 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:26.285788    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfngm\" (UniqueName: \"kubernetes.io/projected/66b1d4fe-410d-47e0-a1a0-c13a71a8d938-kube-api-access-xfngm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:02:26.901181 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:26.901148    4993 generic.go:356] "Generic (PLEG): container finished" podID="66b1d4fe-410d-47e0-a1a0-c13a71a8d938" containerID="cd946b7177645a2624ffd908f90c1ac88458c9667d07d67363fbf7d6c7d024b5" exitCode=137
Nov 03 21:02:26.901380 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:26.901209    4993 scope.go:122] "RemoveContainer" containerID="cd946b7177645a2624ffd908f90c1ac88458c9667d07d67363fbf7d6c7d024b5"
Nov 03 21:02:26.901380 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:26.901218    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-2ff420"
Nov 03 21:02:26.903701 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:26.903605    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 21:02:26.903701 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:26.903680    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr" podUID="82ef457f-b231-432d-9c89-52e22180cea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 21:02:26.911435 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:26.911416    4993 scope.go:122] "RemoveContainer" containerID="cd946b7177645a2624ffd908f90c1ac88458c9667d07d67363fbf7d6c7d024b5"
Nov 03 21:02:26.911792 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:02:26.911768    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd946b7177645a2624ffd908f90c1ac88458c9667d07d67363fbf7d6c7d024b5\": container with ID starting with cd946b7177645a2624ffd908f90c1ac88458c9667d07d67363fbf7d6c7d024b5 not found: ID does not exist" containerID="cd946b7177645a2624ffd908f90c1ac88458c9667d07d67363fbf7d6c7d024b5"
Nov 03 21:02:26.911879 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:26.911803    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd946b7177645a2624ffd908f90c1ac88458c9667d07d67363fbf7d6c7d024b5"} err="failed to get container status \"cd946b7177645a2624ffd908f90c1ac88458c9667d07d67363fbf7d6c7d024b5\": rpc error: code = NotFound desc = could not find container \"cd946b7177645a2624ffd908f90c1ac88458c9667d07d67363fbf7d6c7d024b5\": container with ID starting with cd946b7177645a2624ffd908f90c1ac88458c9667d07d67363fbf7d6c7d024b5 not found: ID does not exist"
Nov 03 21:02:26.960878 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:02:26.960831    4993 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7364-2ff420\" is forbidden: User \"system:node:ip-10-0-142-223.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-223.us-west-1.compute.internal' and this object" podUID="66b1d4fe-410d-47e0-a1a0-c13a71a8d938" pod="opendatahub/test-curl-7364-2ff420"
Nov 03 21:02:27.602470 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:27.602434    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="66b1d4fe-410d-47e0-a1a0-c13a71a8d938" path="/var/lib/kubelet/pods/66b1d4fe-410d-47e0-a1a0-c13a71a8d938/volumes"
Nov 03 21:02:27.642488 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:02:27.642450    4993 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7364-2ff420\" is forbidden: User \"system:node:ip-10-0-142-223.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-223.us-west-1.compute.internal' and this object" podUID="66b1d4fe-410d-47e0-a1a0-c13a71a8d938" pod="opendatahub/test-curl-7364-2ff420"
Nov 03 21:02:27.903773 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:27.903679    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 21:02:27.903773 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:27.903748    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr" podUID="82ef457f-b231-432d-9c89-52e22180cea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 21:02:28.903225 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:28.903190    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Nov 03 21:02:28.903604 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:28.903264    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr" podUID="82ef457f-b231-432d-9c89-52e22180cea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Nov 03 21:02:30.358420 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:30.358351    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:30.360182 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:30.360130    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:30.370946 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:30.370915    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:02:31.673880 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:02:31.673844    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:02:44.441662 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:44.441062    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:44.662055 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:44.662005    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:44.694880 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:44.694762    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 03 21:02:44.802656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:44.801097    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:44.853257 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:44.853212    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:44.854852 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:44.853522    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.038658 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.035732    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.057782 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.057732    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.248571 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.248526    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.293927 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.293743    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.421248 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.420836    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.443898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.443845    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.444422 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.444273    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 21:02:45.452819 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.452773    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.453932 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.453905    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": context deadline exceeded"
Nov 03 21:02:45.670715 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.636769    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 03 21:02:45.694657 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.680035    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.695420 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.695377    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.813678 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.807005    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.858856 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.858815    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.859373 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.859341    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:45.940739 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.940617    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:02:45.941009 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:45.940971    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m" podUID="68cccb32-d2a6-48f7-98f6-f0f60aab5496" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:46.037916 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.037868    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:46.068336 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.065727    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:46.159660 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.147048    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:46.575656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.499080    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:46.575656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.521171    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": context deadline exceeded"
Nov 03 21:02:46.575656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.552106    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:02:46.575656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.552173    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m" podUID="529e3166-0d8e-40e1-93a7-a6b563ce5c8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:46.689961 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.689912    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:46.691141 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.691093    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 03 21:02:46.707660 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.696256    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": context deadline exceeded"
Nov 03 21:02:46.728029 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:02:46.727978    4993 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.132s"
Nov 03 21:02:46.789080 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.789049    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 21:02:46.799933 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.757035    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 03 21:02:46.871333 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.863964    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 21:02:46.885982 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.885937    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 03 21:02:46.937472 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.903783    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Nov 03 21:02:46.937472 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.923915    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 21:02:46.946065 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.945541    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:47.038656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:46.947157    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 21:02:47.083792 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.074871    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:47.193711 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.191752    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 03 21:02:47.193711 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.191840    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:47.193711 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.191958    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": context deadline exceeded"
Nov 03 21:02:47.193711 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.191997    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:02:47.326209 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.323777    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 21:02:47.326209 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.323868    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 21:02:47.326209 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.323909    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 21:02:47.326209 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.323944    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 21:02:47.326209 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.323983    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 21:02:47.326209 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.324017    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:02:47.326209 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.324054    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:02:47.326209 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.324089    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 21:02:47.326209 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.324106    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 21:02:47.326209 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.324139    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 21:02:47.326209 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.324169    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:02:47.346910 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.346882    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 21:02:47.359792 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.359764    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 21:02:47.361128 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.361103    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 21:02:47.370660 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.370019    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 21:02:47.372067 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.372048    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 21:02:47.385292 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.385272    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 21:02:47.387001 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.386984    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 21:02:47.391178 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.391162    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 21:02:47.400587 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.400570    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:02:47.408612 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:47.408595    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 21:02:58.151340 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.151254    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"]
Nov 03 21:02:58.151865 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.151848    4993 cpu_manager.go:414] "containerMap: removing container" podUID="66b1d4fe-410d-47e0-a1a0-c13a71a8d938" containerName="test-curl-7364-2ff420"
Nov 03 21:02:58.151917 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.151867    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66b1d4fe-410d-47e0-a1a0-c13a71a8d938" containerName="test-curl-7364-2ff420"
Nov 03 21:02:58.152010 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.152001    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="66b1d4fe-410d-47e0-a1a0-c13a71a8d938" containerName="test-curl-7364-2ff420"
Nov 03 21:02:58.156578 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.156554    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.209517 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.209484    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 03 21:02:58.209751 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.209528    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 03 21:02:58.209751 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.209536    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 03 21:02:58.258747 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.258717    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cqrrn\""
Nov 03 21:02:58.262050 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.262021    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"]
Nov 03 21:02:58.277399 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.277357    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.277560 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.277484    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.277668 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.277575    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccv9m\" (UniqueName: \"kubernetes.io/projected/293bb466-4d73-49bf-aa53-cf512e226e5b-kube-api-access-ccv9m\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.277668 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.277621    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.277803 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.277665    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/293bb466-4d73-49bf-aa53-cf512e226e5b-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.277803 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.277721    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.277924 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.277814    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.379079 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.379039    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.379272 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.379162    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.379341 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.379300    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.379397 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.379369    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.379452 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.379438    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccv9m\" (UniqueName: \"kubernetes.io/projected/293bb466-4d73-49bf-aa53-cf512e226e5b-kube-api-access-ccv9m\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.379502 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.379479    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.379566 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.379504    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/293bb466-4d73-49bf-aa53-cf512e226e5b-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.379900 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.379868    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.382583 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.382549    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.382783 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.382760    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/293bb466-4d73-49bf-aa53-cf512e226e5b-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.382929 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.382907    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.384014 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.383990    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.446197 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.446134    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccv9m\" (UniqueName: \"kubernetes.io/projected/293bb466-4d73-49bf-aa53-cf512e226e5b-kube-api-access-ccv9m\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.447428 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.447409    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-56nxk\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.470486 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.470455    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:58.911518 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.911481    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"]
Nov 03 21:02:58.912377 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:02:58.912352    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod293bb466_4d73_49bf_aa53_cf512e226e5b.slice/crio-eaaf69050efb2b252764ba6f2b62871bb8b81586410bcd3d5f9d0a7d10cea525 WatchSource:0}: Error finding container eaaf69050efb2b252764ba6f2b62871bb8b81586410bcd3d5f9d0a7d10cea525: Status 404 returned error can't find the container with id eaaf69050efb2b252764ba6f2b62871bb8b81586410bcd3d5f9d0a7d10cea525
Nov 03 21:02:58.915300 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.915266    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:02:58.915383 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:58.915336    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:02:59.431704 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:59.431660    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk" event={"ID":"293bb466-4d73-49bf-aa53-cf512e226e5b","Type":"ContainerStarted","Data":"212c89824de35072bb7651ba1a6d89da153d215f3b4350ff4a7bc6d711257772"}
Nov 03 21:02:59.432165 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:59.431715    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk" event={"ID":"293bb466-4d73-49bf-aa53-cf512e226e5b","Type":"ContainerStarted","Data":"eaaf69050efb2b252764ba6f2b62871bb8b81586410bcd3d5f9d0a7d10cea525"}
Nov 03 21:02:59.432165 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:59.431888    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:02:59.433489 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:59.433462    4993 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-56nxk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:02:59.433654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:02:59.433510    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk" podUID="293bb466-4d73-49bf-aa53-cf512e226e5b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:03:00.449683 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:00.449492    4993 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-56nxk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:03:00.449683 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:00.449546    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk" podUID="293bb466-4d73-49bf-aa53-cf512e226e5b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:03:01.671699 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:03:01.671658    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:03:03.449290 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:03.449257    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:03:03.669205 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:03.669148    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk" podStartSLOduration=5.669135884 podStartE2EDuration="5.669135884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:02:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:02:59.592250257 +0000 UTC m=+1382.136956627" watchObservedRunningTime="2026-11-03 21:03:03.669135884 +0000 UTC m=+1386.213842254"
Nov 03 21:03:08.281663 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.281606    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"]
Nov 03 21:03:08.285856 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.285833    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.349317 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.349288    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.349488 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.349326    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.349488 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.349357    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.349578 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.349538    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.349578 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.349567    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.349703 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.349603    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.349766 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.349751    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9s5x\" (UniqueName: \"kubernetes.io/projected/45229f7a-8c58-4b55-b532-ff4aaf5032dd-kube-api-access-n9s5x\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.349802 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.349790    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.349839 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.349823    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.390073 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.390040    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"]
Nov 03 21:03:08.451467 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.451435    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.451718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.451476    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.451718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.451529    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.451718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.451588    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9s5x\" (UniqueName: \"kubernetes.io/projected/45229f7a-8c58-4b55-b532-ff4aaf5032dd-kube-api-access-n9s5x\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.451718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.451661    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.451953 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.451919    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.452051 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.452019    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-workload-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.452121 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.452054    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.452121 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.452083    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.452229 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.452120    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.452229 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.452185    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-credential-socket\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.452470 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.452448    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-workload-certs\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.452579 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.452467    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istiod-ca-cert\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.452698 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.452679    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-data\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.454311 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.454291    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-envoy\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.454463 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.454444    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-podinfo\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.513061 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.513032    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-token\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.513820 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.513795    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9s5x\" (UniqueName: \"kubernetes.io/projected/45229f7a-8c58-4b55-b532-ff4aaf5032dd-kube-api-access-n9s5x\") pod \"e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") " pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:08.606505 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:08.606413    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:09.046746 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:09.046716    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"]
Nov 03 21:03:09.048188 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:03:09.048160    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45229f7a_8c58_4b55_b532_ff4aaf5032dd.slice/crio-f26dda8b56692638bfd05e53309ec4ac47a319b6fae097876fcccd0656c55eaf WatchSource:0}: Error finding container f26dda8b56692638bfd05e53309ec4ac47a319b6fae097876fcccd0656c55eaf: Status 404 returned error can't find the container with id f26dda8b56692638bfd05e53309ec4ac47a319b6fae097876fcccd0656c55eaf
Nov 03 21:03:09.051350 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:09.051317    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:09.051420 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:09.051385    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:09.051458 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:09.051419    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:09.512783 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:09.512744    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5" event={"ID":"45229f7a-8c58-4b55-b532-ff4aaf5032dd","Type":"ContainerStarted","Data":"0f95201ab6be0289048420dbacee187c3f116079901bec5b67eae5daf71ce481"}
Nov 03 21:03:09.513253 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:09.512806    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5" event={"ID":"45229f7a-8c58-4b55-b532-ff4aaf5032dd","Type":"ContainerStarted","Data":"f26dda8b56692638bfd05e53309ec4ac47a319b6fae097876fcccd0656c55eaf"}
Nov 03 21:03:09.608176 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:09.607368    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:09.636222 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:09.636168    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5" podStartSLOduration=1.636151752 podStartE2EDuration="1.636151752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:03:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:03:09.635149637 +0000 UTC m=+1392.179856008" watchObservedRunningTime="2026-11-03 21:03:09.636151752 +0000 UTC m=+1392.180858209"
Nov 03 21:03:10.608935 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:10.608901    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:03:10.609412 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:10.608964    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5" podUID="45229f7a-8c58-4b55-b532-ff4aaf5032dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:03:11.607350 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:11.607316    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:03:11.607545 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:11.607366    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5" podUID="45229f7a-8c58-4b55-b532-ff4aaf5032dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:03:12.607336 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:12.607279    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:03:12.607888 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:12.607364    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5" podUID="45229f7a-8c58-4b55-b532-ff4aaf5032dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:03:13.607043 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:13.607007    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:03:13.607230 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:13.607065    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5" podUID="45229f7a-8c58-4b55-b532-ff4aaf5032dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:03:14.677655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:14.677424    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:03:14.677655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:14.677489    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5" podUID="45229f7a-8c58-4b55-b532-ff4aaf5032dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:03:15.606797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:15.606762    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Nov 03 21:03:15.607010 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:15.606825    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5" podUID="45229f7a-8c58-4b55-b532-ff4aaf5032dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Nov 03 21:03:17.010411 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:17.010373    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:17.010893 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:17.010812    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:17.011530 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:17.011501    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:03:31.897540 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:03:31.897496    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:03:32.200720 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:32.198218    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:32.383654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:32.382734    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 03 21:03:32.415655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:32.413423    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 03 21:03:32.636809 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:32.636761    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:32.673733 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:32.673679    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:33.099784 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.099742    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:03:33.100390 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.100357    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" podUID="0ceb5aa1-a956-4442-b710-36453c7a1e79" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:33.246654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.245891    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:03:33.246654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.245956    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" podUID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:33.273674 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.273090    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:33.327657 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.325727    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:33.354848 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.354743    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": context deadline exceeded"
Nov 03 21:03:33.375769 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.375718    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:33.405653 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.399725    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:33.405653 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.400126    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 03 21:03:33.405653 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.400422    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:33.405653 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.400718    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": context deadline exceeded"
Nov 03 21:03:33.405653 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.401967    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 21:03:33.491354 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.488736    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded"
Nov 03 21:03:33.491354 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.489222    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:03:33.558656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.515079    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": context deadline exceeded"
Nov 03 21:03:33.558656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.515431    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:33.558656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.515730    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Nov 03 21:03:33.797746 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.709722    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:33.797746 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:33.710147    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:34.192139 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.174082    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 21:03:34.192768 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.192746    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:03:34.238662 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.232052    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 21:03:34.360502 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.359738    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:34.380666 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.365290    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 21:03:34.380666 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.372747    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:34.380666 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.373139    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 21:03:34.382805 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.381156    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:34.382805 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.381475    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 21:03:34.447342 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.434058    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:34.462974 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.445808    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:34.464295 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.445865    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": context deadline exceeded"
Nov 03 21:03:34.495733 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.472984    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 21:03:34.495733 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.473045    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 21:03:34.497446 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.496164    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:34.497446 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.497040    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 21:03:34.502111 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.500857    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:34.537107 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.533821    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": context deadline exceeded"
Nov 03 21:03:34.537107 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.534284    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 03 21:03:34.537107 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.534514    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": context deadline exceeded"
Nov 03 21:03:34.537107 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.534772    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 21:03:34.537107 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.534834    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 21:03:34.537107 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.534871    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 21:03:34.567205 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.562224    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/health\": context deadline exceeded"
Nov 03 21:03:34.710723 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.710502    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": context deadline exceeded"
Nov 03 21:03:35.000708 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:34.997153    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 21:03:35.325710 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.323867    4993 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:35.357986 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.357954    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 21:03:35.360791 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.360756    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": context deadline exceeded"
Nov 03 21:03:35.373446 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.373405    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:35.384918 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.384880    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:35.449761 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.449714    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:35.495351 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.495312    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:03:35.501332 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.501282    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:35.504650 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.501547    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" probeResult="failure" output="Get \"https://10.133.0.41:8000/ready\": context deadline exceeded"
Nov 03 21:03:35.542674 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.536080    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:03:35.542674 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.536440    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" probeResult="failure" output="Get \"https://10.133.0.45:8000/ready\": context deadline exceeded"
Nov 03 21:03:35.542674 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.536755    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": context deadline exceeded"
Nov 03 21:03:35.606149 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.606064    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 21:03:35.752365 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.752320    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 21:03:35.798654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.797382    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 21:03:35.798654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.797602    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 21:03:35.798654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.797766    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:03:35.822669 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.821914    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 21:03:35.844955 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.844923    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 21:03:35.937084 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.936998    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 21:03:35.952769 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:35.952740    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 21:03:53.244325 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.244285    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"]
Nov 03 21:03:53.248373 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.248351    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.361268 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.361165    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.361541 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.361511    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.361711 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.361660    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4sjk\" (UniqueName: \"kubernetes.io/projected/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-kube-api-access-q4sjk\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.361794 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.361747    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.361794 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.361787    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.361916 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.361877    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.361916 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.361906    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.362029 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.361932    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.362029 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.361980    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.454734 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.454701    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"]
Nov 03 21:03:53.463615 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.463583    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.463805 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.463688    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.463805 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.463718    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.463805 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.463744    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.463805 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.463791    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.464034 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.463930    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.464034 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.463966    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.464149 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.464084    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4sjk\" (UniqueName: \"kubernetes.io/projected/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-kube-api-access-q4sjk\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.464200 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.464170    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.464866 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.464834    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.465252 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.465229    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.465407 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.465378    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.465664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.465622    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.465978 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.465957    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.468140 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.468115    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.468730 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.468703    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.528922 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.528842    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4sjk\" (UniqueName: \"kubernetes.io/projected/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-kube-api-access-q4sjk\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.528922 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.528899    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-g24jg\" (UID: \"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:53.564027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:53.563995    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:54.001939 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:54.001909    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"]
Nov 03 21:03:54.003210 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:03:54.003182    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad5c0f32_0285_4b4b_bf4a_db9ef6b12d1b.slice/crio-e3e6fea437ac42715e516ecf7c0fcd740b8e340972c206e15b3694dec02c7ccb WatchSource:0}: Error finding container e3e6fea437ac42715e516ecf7c0fcd740b8e340972c206e15b3694dec02c7ccb: Status 404 returned error can't find the container with id e3e6fea437ac42715e516ecf7c0fcd740b8e340972c206e15b3694dec02c7ccb
Nov 03 21:03:54.005820 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:54.005791    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:54.005896 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:54.005860    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:54.005896 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:54.005888    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:03:54.381601 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:54.381495    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg" event={"ID":"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b","Type":"ContainerStarted","Data":"840fbd421d8e01a5ce523cca05620b6f17e04795d1e9043d43b126955e30d215"}
Nov 03 21:03:54.381601 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:54.381570    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg" event={"ID":"ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b","Type":"ContainerStarted","Data":"e3e6fea437ac42715e516ecf7c0fcd740b8e340972c206e15b3694dec02c7ccb"}
Nov 03 21:03:54.503227 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:54.503165    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg" podStartSLOduration=1.503150749 podStartE2EDuration="1.503150749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:03:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:03:54.502654801 +0000 UTC m=+1437.047361171" watchObservedRunningTime="2026-11-03 21:03:54.503150749 +0000 UTC m=+1437.047857112"
Nov 03 21:03:54.565330 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:54.564733    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:03:55.569757 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:55.569721    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:03:55.588764 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:55.588720    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg" podUID="ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:03:56.567654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:56.566957    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:03:56.567654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:56.567013    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg" podUID="ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:03:57.593859 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:57.593821    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:03:57.594405 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:57.593880    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg" podUID="ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:03:58.585080 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:58.585038    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:03:58.585502 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:58.585474    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg" podUID="ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:03:59.567871 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:59.567832    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:03:59.568491 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:03:59.568456    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg" podUID="ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:04:00.564430 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:00.564392    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:04:00.564685 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:00.564453    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg" podUID="ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:04:01.564836 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:01.564805    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Nov 03 21:04:01.565315 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:01.564858    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg" podUID="ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Nov 03 21:04:01.670467 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:04:01.670418    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:04:02.928682 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:02.927031    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:04:02.929510 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:02.929488    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:04:02.931217 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:02.931180    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg"
Nov 03 21:04:08.365823 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.365710    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"]
Nov 03 21:04:08.370993 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.370967    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.477214 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.477180    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"]
Nov 03 21:04:08.519361 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.519321    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b056b057-0700-4450-b07d-f19b53d5584a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.519572 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.519482    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b056b057-0700-4450-b07d-f19b53d5584a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.519694 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.519668    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b056b057-0700-4450-b07d-f19b53d5584a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.519756 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.519721    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b056b057-0700-4450-b07d-f19b53d5584a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.519806 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.519764    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b056b057-0700-4450-b07d-f19b53d5584a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.519905 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.519884    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b056b057-0700-4450-b07d-f19b53d5584a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.519968 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.519954    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b056b057-0700-4450-b07d-f19b53d5584a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.520088 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.520068    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fzm9\" (UniqueName: \"kubernetes.io/projected/b056b057-0700-4450-b07d-f19b53d5584a-kube-api-access-2fzm9\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.520133 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.520113    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b056b057-0700-4450-b07d-f19b53d5584a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.622017 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.621934    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fzm9\" (UniqueName: \"kubernetes.io/projected/b056b057-0700-4450-b07d-f19b53d5584a-kube-api-access-2fzm9\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.622017 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.621974    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b056b057-0700-4450-b07d-f19b53d5584a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.622228 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.622105    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b056b057-0700-4450-b07d-f19b53d5584a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.622228 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.622216    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b056b057-0700-4450-b07d-f19b53d5584a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.622347 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.622329    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b056b057-0700-4450-b07d-f19b53d5584a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.622396 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.622371    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b056b057-0700-4450-b07d-f19b53d5584a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.622458 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.622399    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b056b057-0700-4450-b07d-f19b53d5584a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.622458 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.622442    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b056b057-0700-4450-b07d-f19b53d5584a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.622556 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.622485    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b056b057-0700-4450-b07d-f19b53d5584a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.622556 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.622495    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b056b057-0700-4450-b07d-f19b53d5584a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.622556 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.622522    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b056b057-0700-4450-b07d-f19b53d5584a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.622731 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.622678    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b056b057-0700-4450-b07d-f19b53d5584a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.622888 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.622781    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b056b057-0700-4450-b07d-f19b53d5584a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.623231 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.623200    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b056b057-0700-4450-b07d-f19b53d5584a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.624581 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.624553    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b056b057-0700-4450-b07d-f19b53d5584a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.625298 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.625275    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b056b057-0700-4450-b07d-f19b53d5584a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.682959 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.682929    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b056b057-0700-4450-b07d-f19b53d5584a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.683072 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.682979    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fzm9\" (UniqueName: \"kubernetes.io/projected/b056b057-0700-4450-b07d-f19b53d5584a-kube-api-access-2fzm9\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7\" (UID: \"b056b057-0700-4450-b07d-f19b53d5584a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:08.685927 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:08.685906    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:09.118183 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:09.118145    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"]
Nov 03 21:04:09.154818 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:04:09.154789    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb056b057_0700_4450_b07d_f19b53d5584a.slice/crio-7896c8876fa0aa89954bb3ab53cc96e88c7b490eeb1a03e635035f3135422efe WatchSource:0}: Error finding container 7896c8876fa0aa89954bb3ab53cc96e88c7b490eeb1a03e635035f3135422efe: Status 404 returned error can't find the container with id 7896c8876fa0aa89954bb3ab53cc96e88c7b490eeb1a03e635035f3135422efe
Nov 03 21:04:09.157575 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:09.157541    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:04:09.157706 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:09.157612    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:04:09.157706 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:09.157662    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:04:09.503669 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:09.503609    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7" event={"ID":"b056b057-0700-4450-b07d-f19b53d5584a","Type":"ContainerStarted","Data":"571a4a3ae9739b4a09e7765e9874725fb18f59d2fd06d38d7bb6108d607b50f3"}
Nov 03 21:04:09.504159 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:09.503697    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7" event={"ID":"b056b057-0700-4450-b07d-f19b53d5584a","Type":"ContainerStarted","Data":"7896c8876fa0aa89954bb3ab53cc96e88c7b490eeb1a03e635035f3135422efe"}
Nov 03 21:04:09.648322 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:09.648256    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7" podStartSLOduration=1.6482384780000001 podStartE2EDuration="1.648238478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:04:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:04:09.646593112 +0000 UTC m=+1452.191299482" watchObservedRunningTime="2026-11-03 21:04:09.648238478 +0000 UTC m=+1452.192944848"
Nov 03 21:04:09.695654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:09.691730    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:10.698656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:10.696201    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 03 21:04:10.698656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:10.696290    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7" podUID="b056b057-0700-4450-b07d-f19b53d5584a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 03 21:04:11.691053 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:11.691016    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 03 21:04:11.693011 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:11.692974    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7" podUID="b056b057-0700-4450-b07d-f19b53d5584a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 03 21:04:12.698681 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:12.696389    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 03 21:04:12.698681 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:12.696451    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7" podUID="b056b057-0700-4450-b07d-f19b53d5584a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 03 21:04:13.688033 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:13.687996    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 03 21:04:13.688308 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:13.688280    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7" podUID="b056b057-0700-4450-b07d-f19b53d5584a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 03 21:04:14.696658 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:14.692139    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 03 21:04:14.696658 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:14.692201    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7" podUID="b056b057-0700-4450-b07d-f19b53d5584a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 03 21:04:15.687132 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:15.687091    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 03 21:04:15.687329 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:15.687167    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7" podUID="b056b057-0700-4450-b07d-f19b53d5584a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 03 21:04:16.686971 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:16.686915    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Nov 03 21:04:16.687478 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:16.686994    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7" podUID="b056b057-0700-4450-b07d-f19b53d5584a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Nov 03 21:04:18.071252 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:18.071222    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:18.071779 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:18.071754    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:18.072615 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:04:18.072592    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7"
Nov 03 21:04:31.672095 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:04:31.672058    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:05:01.674998 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:05:01.674957    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:05:31.672843 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:05:31.672804    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:05:52.889315 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:52.889213    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"]
Nov 03 21:05:52.889940 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:52.889486    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f" podUID="76814bbc-dc45-440a-947c-1f4979ad86de" containerName="istio-proxy" containerID="cri-o://b483504c16bb5cd44be0e00309d1b16e4e0d18147b79590a4956ea706f8da13d" gracePeriod=30
Nov 03 21:05:58.173852 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.173784    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:05:58.314841 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.314802    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-istio-envoy\") pod \"76814bbc-dc45-440a-947c-1f4979ad86de\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") "
Nov 03 21:05:58.315013 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.314881    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-istio-data\") pod \"76814bbc-dc45-440a-947c-1f4979ad86de\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") "
Nov 03 21:05:58.315013 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.314904    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-workload-certs\") pod \"76814bbc-dc45-440a-947c-1f4979ad86de\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") "
Nov 03 21:05:58.315013 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.314963    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/76814bbc-dc45-440a-947c-1f4979ad86de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76814bbc-dc45-440a-947c-1f4979ad86de-istio-podinfo\") pod \"76814bbc-dc45-440a-947c-1f4979ad86de\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") "
Nov 03 21:05:58.315013 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.314991    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-workload-socket\") pod \"76814bbc-dc45-440a-947c-1f4979ad86de\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") "
Nov 03 21:05:58.315229 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.315027    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76814bbc-dc45-440a-947c-1f4979ad86de-istio-token\" (UniqueName: \"kubernetes.io/projected/76814bbc-dc45-440a-947c-1f4979ad86de-istio-token\") pod \"76814bbc-dc45-440a-947c-1f4979ad86de\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") "
Nov 03 21:05:58.315229 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.315131    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-istio-data" pod "76814bbc-dc45-440a-947c-1f4979ad86de" (UID: "76814bbc-dc45-440a-947c-1f4979ad86de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:05:58.315229 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.315224    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-credential-socket\") pod \"76814bbc-dc45-440a-947c-1f4979ad86de\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") "
Nov 03 21:05:58.315399 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.315246    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-workload-certs" pod "76814bbc-dc45-440a-947c-1f4979ad86de" (UID: "76814bbc-dc45-440a-947c-1f4979ad86de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:05:58.315399 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.315326    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-workload-socket" pod "76814bbc-dc45-440a-947c-1f4979ad86de" (UID: "76814bbc-dc45-440a-947c-1f4979ad86de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:05:58.315399 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.315339    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76814bbc-dc45-440a-947c-1f4979ad86de-kube-api-access-4fr4r\" (UniqueName: \"kubernetes.io/projected/76814bbc-dc45-440a-947c-1f4979ad86de-kube-api-access-4fr4r\") pod \"76814bbc-dc45-440a-947c-1f4979ad86de\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") "
Nov 03 21:05:58.315572 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.315427    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76814bbc-dc45-440a-947c-1f4979ad86de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76814bbc-dc45-440a-947c-1f4979ad86de-istiod-ca-cert\") pod \"76814bbc-dc45-440a-947c-1f4979ad86de\" (UID: \"76814bbc-dc45-440a-947c-1f4979ad86de\") "
Nov 03 21:05:58.315572 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.315547    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-credential-socket" pod "76814bbc-dc45-440a-947c-1f4979ad86de" (UID: "76814bbc-dc45-440a-947c-1f4979ad86de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:05:58.315896 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.315859    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76814bbc-dc45-440a-947c-1f4979ad86de-istiod-ca-cert" pod "76814bbc-dc45-440a-947c-1f4979ad86de" (UID: "76814bbc-dc45-440a-947c-1f4979ad86de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:05:58.316298 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.316275    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:58.316414 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.316303    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:58.316414 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.316321    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:58.316414 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.316337    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:58.316414 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.316351    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76814bbc-dc45-440a-947c-1f4979ad86de-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:58.317592 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.317565    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-istio-envoy" pod "76814bbc-dc45-440a-947c-1f4979ad86de" (UID: "76814bbc-dc45-440a-947c-1f4979ad86de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:05:58.317592 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.317581    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76814bbc-dc45-440a-947c-1f4979ad86de-istio-token" pod "76814bbc-dc45-440a-947c-1f4979ad86de" (UID: "76814bbc-dc45-440a-947c-1f4979ad86de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:05:58.318250 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.318220    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76814bbc-dc45-440a-947c-1f4979ad86de-istio-podinfo" pod "76814bbc-dc45-440a-947c-1f4979ad86de" (UID: "76814bbc-dc45-440a-947c-1f4979ad86de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:05:58.318360 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.318232    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76814bbc-dc45-440a-947c-1f4979ad86de-kube-api-access-4fr4r" pod "76814bbc-dc45-440a-947c-1f4979ad86de" (UID: "76814bbc-dc45-440a-947c-1f4979ad86de"). InnerVolumeSpecName "kube-api-access-4fr4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:05:58.417801 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.417768    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76814bbc-dc45-440a-947c-1f4979ad86de-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:58.417801 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.417795    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76814bbc-dc45-440a-947c-1f4979ad86de-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:58.417801 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.417806    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fr4r\" (UniqueName: \"kubernetes.io/projected/76814bbc-dc45-440a-947c-1f4979ad86de-kube-api-access-4fr4r\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:58.418020 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.417817    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76814bbc-dc45-440a-947c-1f4979ad86de-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:05:58.661552 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.661515    4993 generic.go:356] "Generic (PLEG): container finished" podID="76814bbc-dc45-440a-947c-1f4979ad86de" containerID="b483504c16bb5cd44be0e00309d1b16e4e0d18147b79590a4956ea706f8da13d" exitCode=0
Nov 03 21:05:58.661793 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.661568    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f" event={"ID":"76814bbc-dc45-440a-947c-1f4979ad86de","Type":"ContainerDied","Data":"b483504c16bb5cd44be0e00309d1b16e4e0d18147b79590a4956ea706f8da13d"}
Nov 03 21:05:58.661793 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.661572    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"
Nov 03 21:05:58.661793 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.661612    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f" event={"ID":"76814bbc-dc45-440a-947c-1f4979ad86de","Type":"ContainerDied","Data":"bf329afd71390f631502085c216e65f3f0d8976a3f16596372ca6f2cb91d3a3d"}
Nov 03 21:05:58.661793 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.661669    4993 scope.go:122] "RemoveContainer" containerID="b483504c16bb5cd44be0e00309d1b16e4e0d18147b79590a4956ea706f8da13d"
Nov 03 21:05:58.680130 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.680109    4993 scope.go:122] "RemoveContainer" containerID="b483504c16bb5cd44be0e00309d1b16e4e0d18147b79590a4956ea706f8da13d"
Nov 03 21:05:58.680608 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:05:58.680580    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b483504c16bb5cd44be0e00309d1b16e4e0d18147b79590a4956ea706f8da13d\": container with ID starting with b483504c16bb5cd44be0e00309d1b16e4e0d18147b79590a4956ea706f8da13d not found: ID does not exist" containerID="b483504c16bb5cd44be0e00309d1b16e4e0d18147b79590a4956ea706f8da13d"
Nov 03 21:05:58.680743 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.680619    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b483504c16bb5cd44be0e00309d1b16e4e0d18147b79590a4956ea706f8da13d"} err="failed to get container status \"b483504c16bb5cd44be0e00309d1b16e4e0d18147b79590a4956ea706f8da13d\": rpc error: code = NotFound desc = could not find container \"b483504c16bb5cd44be0e00309d1b16e4e0d18147b79590a4956ea706f8da13d\": container with ID starting with b483504c16bb5cd44be0e00309d1b16e4e0d18147b79590a4956ea706f8da13d not found: ID does not exist"
Nov 03 21:05:58.834975 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.834943    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"]
Nov 03 21:05:58.884266 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:58.884227    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-ca4642-openshift-default-6b89f76569-tnt8f"]
Nov 03 21:05:59.579003 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:05:59.578967    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76814bbc-dc45-440a-947c-1f4979ad86de" path="/var/lib/kubelet/pods/76814bbc-dc45-440a-947c-1f4979ad86de/volumes"
Nov 03 21:06:01.670318 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:06:01.670282    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:06:31.672559 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:06:31.672484    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:07:01.672574 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:07:01.672535    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:07:31.673250 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:07:31.673206    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:07:39.737835 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:39.737795    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"]
Nov 03 21:07:39.738437 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:39.738013    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46" podUID="5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" containerName="istio-proxy" containerID="cri-o://adb77385ab75a80eda9faa7d63f659dae808b1833f9717052050e00f4645c03c" gracePeriod=30
Nov 03 21:07:39.787189 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:39.787150    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"]
Nov 03 21:07:39.787402 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:39.787376    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj" podUID="260b8942-a2ee-41fa-b64d-daa30ecc0bc1" containerName="istio-proxy" containerID="cri-o://59e81033fc9b04024f9a73d70a58fbfdd43d1365477b90d4389e88975f4f57a3" gracePeriod=30
Nov 03 21:07:45.201390 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.201362    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:07:45.207242 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.206137    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:07:45.308546 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.308456    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-kube-api-access-27r2c\" (UniqueName: \"kubernetes.io/projected/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-kube-api-access-27r2c\") pod \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") "
Nov 03 21:07:45.308546 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.308498    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-kube-api-access-ptfrr\" (UniqueName: \"kubernetes.io/projected/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-kube-api-access-ptfrr\") pod \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") "
Nov 03 21:07:45.308837 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.308578    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-workload-certs\") pod \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") "
Nov 03 21:07:45.308837 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.308731    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-workload-socket\") pod \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") "
Nov 03 21:07:45.308837 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.308807    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istiod-ca-cert\") pod \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") "
Nov 03 21:07:45.309002 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.308839    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-podinfo\") pod \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") "
Nov 03 21:07:45.309002 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.308876    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-data\") pod \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") "
Nov 03 21:07:45.309002 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.308914    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-workload-certs" pod "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" (UID: "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:45.309002 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.308927    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-envoy\") pod \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") "
Nov 03 21:07:45.309237 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.309016    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-token\" (UniqueName: \"kubernetes.io/projected/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-token\") pod \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") "
Nov 03 21:07:45.309237 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.309069    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-workload-socket" pod "260b8942-a2ee-41fa-b64d-daa30ecc0bc1" (UID: "260b8942-a2ee-41fa-b64d-daa30ecc0bc1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:45.309237 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.309153    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-token\" (UniqueName: \"kubernetes.io/projected/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-token\") pod \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") "
Nov 03 21:07:45.309237 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.309211    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istiod-ca-cert\") pod \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") "
Nov 03 21:07:45.309479 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.309239    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-workload-socket\") pod \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") "
Nov 03 21:07:45.309479 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.309282    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-data" pod "260b8942-a2ee-41fa-b64d-daa30ecc0bc1" (UID: "260b8942-a2ee-41fa-b64d-daa30ecc0bc1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:45.309479 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.309412    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-envoy\") pod \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") "
Nov 03 21:07:45.309904 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.309486    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-data\") pod \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") "
Nov 03 21:07:45.309904 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.309529    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-podinfo\") pod \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") "
Nov 03 21:07:45.309904 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.309595    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-workload-certs\") pod \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") "
Nov 03 21:07:45.309904 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.309673    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-credential-socket\") pod \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\" (UID: \"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc\") "
Nov 03 21:07:45.309904 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.309763    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-credential-socket\") pod \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\" (UID: \"260b8942-a2ee-41fa-b64d-daa30ecc0bc1\") "
Nov 03 21:07:45.310617 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.310425    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-data" pod "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" (UID: "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:45.311062 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.310852    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-workload-certs" pod "260b8942-a2ee-41fa-b64d-daa30ecc0bc1" (UID: "260b8942-a2ee-41fa-b64d-daa30ecc0bc1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:45.311062 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.310860    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.311062 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.310903    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.311062 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.310917    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istiod-ca-cert" pod "260b8942-a2ee-41fa-b64d-daa30ecc0bc1" (UID: "260b8942-a2ee-41fa-b64d-daa30ecc0bc1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:07:45.311062 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.310924    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.311062 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.310954    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istiod-ca-cert" pod "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" (UID: "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:07:45.311062 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.310964    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.313255 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.312900    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-credential-socket" pod "260b8942-a2ee-41fa-b64d-daa30ecc0bc1" (UID: "260b8942-a2ee-41fa-b64d-daa30ecc0bc1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:45.313255 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.312928    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-envoy" pod "260b8942-a2ee-41fa-b64d-daa30ecc0bc1" (UID: "260b8942-a2ee-41fa-b64d-daa30ecc0bc1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:45.313418 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.313284    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-kube-api-access-ptfrr" pod "260b8942-a2ee-41fa-b64d-daa30ecc0bc1" (UID: "260b8942-a2ee-41fa-b64d-daa30ecc0bc1"). InnerVolumeSpecName "kube-api-access-ptfrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:45.313418 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.313348    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-token" pod "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" (UID: "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:45.313418 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.313351    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-kube-api-access-27r2c" pod "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" (UID: "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc"). InnerVolumeSpecName "kube-api-access-27r2c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:45.313590 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.313540    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-workload-socket" pod "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" (UID: "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:45.313900 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.313858    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-credential-socket" pod "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" (UID: "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:45.314467 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.314439    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-podinfo" pod "260b8942-a2ee-41fa-b64d-daa30ecc0bc1" (UID: "260b8942-a2ee-41fa-b64d-daa30ecc0bc1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:07:45.314705 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.314673    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-envoy" pod "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" (UID: "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:45.315668 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.315611    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-podinfo" pod "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" (UID: "5e886cd0-4ff5-4364-8a6b-3124b16e9cdc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:07:45.315961 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.315940    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-token" pod "260b8942-a2ee-41fa-b64d-daa30ecc0bc1" (UID: "260b8942-a2ee-41fa-b64d-daa30ecc0bc1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:45.412317 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.412283    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.412317 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.412314    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.412317 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.412326    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.412536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.412334    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.412536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.412343    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.412536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.412351    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.412536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.412360    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.412536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.412369    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.412536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.412378    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.412536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.412386    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.412536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.412394    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.412536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.412402    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.412536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.412410    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27r2c\" (UniqueName: \"kubernetes.io/projected/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc-kube-api-access-27r2c\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.412536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.412430    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptfrr\" (UniqueName: \"kubernetes.io/projected/260b8942-a2ee-41fa-b64d-daa30ecc0bc1-kube-api-access-ptfrr\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:45.569715 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.565827    4993 generic.go:356] "Generic (PLEG): container finished" podID="260b8942-a2ee-41fa-b64d-daa30ecc0bc1" containerID="59e81033fc9b04024f9a73d70a58fbfdd43d1365477b90d4389e88975f4f57a3" exitCode=0
Nov 03 21:07:45.569715 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.566048    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj" event={"ID":"260b8942-a2ee-41fa-b64d-daa30ecc0bc1","Type":"ContainerDied","Data":"59e81033fc9b04024f9a73d70a58fbfdd43d1365477b90d4389e88975f4f57a3"}
Nov 03 21:07:45.569715 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.566134    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj" event={"ID":"260b8942-a2ee-41fa-b64d-daa30ecc0bc1","Type":"ContainerDied","Data":"418e1f90c4c5b3c794bd3e2ce213aa7b0a81b74467d866337a8b3f057a51f691"}
Nov 03 21:07:45.569715 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.566164    4993 scope.go:122] "RemoveContainer" containerID="59e81033fc9b04024f9a73d70a58fbfdd43d1365477b90d4389e88975f4f57a3"
Nov 03 21:07:45.569715 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.566330    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"
Nov 03 21:07:45.582326 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.582290    4993 generic.go:356] "Generic (PLEG): container finished" podID="5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" containerID="adb77385ab75a80eda9faa7d63f659dae808b1833f9717052050e00f4645c03c" exitCode=0
Nov 03 21:07:45.582451 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.582439    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"
Nov 03 21:07:45.603994 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.603968    4993 scope.go:122] "RemoveContainer" containerID="59e81033fc9b04024f9a73d70a58fbfdd43d1365477b90d4389e88975f4f57a3"
Nov 03 21:07:45.604154 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.604120    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46" event={"ID":"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc","Type":"ContainerDied","Data":"adb77385ab75a80eda9faa7d63f659dae808b1833f9717052050e00f4645c03c"}
Nov 03 21:07:45.604278 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.604162    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46" event={"ID":"5e886cd0-4ff5-4364-8a6b-3124b16e9cdc","Type":"ContainerDied","Data":"437ba0caacdd97d2f9f2d521f8041ca3bb1884b64262b9e01a3c06cc5019249b"}
Nov 03 21:07:45.604341 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:07:45.604324    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59e81033fc9b04024f9a73d70a58fbfdd43d1365477b90d4389e88975f4f57a3\": container with ID starting with 59e81033fc9b04024f9a73d70a58fbfdd43d1365477b90d4389e88975f4f57a3 not found: ID does not exist" containerID="59e81033fc9b04024f9a73d70a58fbfdd43d1365477b90d4389e88975f4f57a3"
Nov 03 21:07:45.604421 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.604349    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59e81033fc9b04024f9a73d70a58fbfdd43d1365477b90d4389e88975f4f57a3"} err="failed to get container status \"59e81033fc9b04024f9a73d70a58fbfdd43d1365477b90d4389e88975f4f57a3\": rpc error: code = NotFound desc = could not find container \"59e81033fc9b04024f9a73d70a58fbfdd43d1365477b90d4389e88975f4f57a3\": container with ID starting with 59e81033fc9b04024f9a73d70a58fbfdd43d1365477b90d4389e88975f4f57a3 not found: ID does not exist"
Nov 03 21:07:45.604421 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.604366    4993 scope.go:122] "RemoveContainer" containerID="adb77385ab75a80eda9faa7d63f659dae808b1833f9717052050e00f4645c03c"
Nov 03 21:07:45.636039 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.636006    4993 scope.go:122] "RemoveContainer" containerID="adb77385ab75a80eda9faa7d63f659dae808b1833f9717052050e00f4645c03c"
Nov 03 21:07:45.636531 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:07:45.636494    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adb77385ab75a80eda9faa7d63f659dae808b1833f9717052050e00f4645c03c\": container with ID starting with adb77385ab75a80eda9faa7d63f659dae808b1833f9717052050e00f4645c03c not found: ID does not exist" containerID="adb77385ab75a80eda9faa7d63f659dae808b1833f9717052050e00f4645c03c"
Nov 03 21:07:45.636681 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.636540    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adb77385ab75a80eda9faa7d63f659dae808b1833f9717052050e00f4645c03c"} err="failed to get container status \"adb77385ab75a80eda9faa7d63f659dae808b1833f9717052050e00f4645c03c\": rpc error: code = NotFound desc = could not find container \"adb77385ab75a80eda9faa7d63f659dae808b1833f9717052050e00f4645c03c\": container with ID starting with adb77385ab75a80eda9faa7d63f659dae808b1833f9717052050e00f4645c03c not found: ID does not exist"
Nov 03 21:07:45.744734 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.744702    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"]
Nov 03 21:07:45.751708 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.751682    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-lzm46"]
Nov 03 21:07:45.862545 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.862462    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"]
Nov 03 21:07:45.866035 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:45.866009    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-8z9rj"]
Nov 03 21:07:47.579265 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:47.579222    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="260b8942-a2ee-41fa-b64d-daa30ecc0bc1" path="/var/lib/kubelet/pods/260b8942-a2ee-41fa-b64d-daa30ecc0bc1/volumes"
Nov 03 21:07:47.579924 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:47.579901    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" path="/var/lib/kubelet/pods/5e886cd0-4ff5-4364-8a6b-3124b16e9cdc/volumes"
Nov 03 21:07:54.086936 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:54.086899    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"]
Nov 03 21:07:54.087414 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:54.087119    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr" podUID="82ef457f-b231-432d-9c89-52e22180cea2" containerName="istio-proxy" containerID="cri-o://ea0bc4938dc734c4343b185809cb4d48f2081f67e46034d706b67b03b9fa6026" gracePeriod=30
Nov 03 21:07:59.545570 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.545541    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:07:59.644104 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.644071    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82ef457f-b231-432d-9c89-52e22180cea2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82ef457f-b231-432d-9c89-52e22180cea2-istiod-ca-cert\") pod \"82ef457f-b231-432d-9c89-52e22180cea2\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") "
Nov 03 21:07:59.644291 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.644118    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-workload-certs\") pod \"82ef457f-b231-432d-9c89-52e22180cea2\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") "
Nov 03 21:07:59.644291 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.644172    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82ef457f-b231-432d-9c89-52e22180cea2-kube-api-access-4mpck\" (UniqueName: \"kubernetes.io/projected/82ef457f-b231-432d-9c89-52e22180cea2-kube-api-access-4mpck\") pod \"82ef457f-b231-432d-9c89-52e22180cea2\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") "
Nov 03 21:07:59.644291 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.644240    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-credential-socket\") pod \"82ef457f-b231-432d-9c89-52e22180cea2\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") "
Nov 03 21:07:59.644470 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.644304    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-istio-envoy\") pod \"82ef457f-b231-432d-9c89-52e22180cea2\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") "
Nov 03 21:07:59.644470 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.644350    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82ef457f-b231-432d-9c89-52e22180cea2-istio-token\" (UniqueName: \"kubernetes.io/projected/82ef457f-b231-432d-9c89-52e22180cea2-istio-token\") pod \"82ef457f-b231-432d-9c89-52e22180cea2\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") "
Nov 03 21:07:59.644470 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.644446    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-workload-certs" pod "82ef457f-b231-432d-9c89-52e22180cea2" (UID: "82ef457f-b231-432d-9c89-52e22180cea2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:59.644617 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.644477    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/82ef457f-b231-432d-9c89-52e22180cea2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82ef457f-b231-432d-9c89-52e22180cea2-istio-podinfo\") pod \"82ef457f-b231-432d-9c89-52e22180cea2\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") "
Nov 03 21:07:59.644617 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.644523    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82ef457f-b231-432d-9c89-52e22180cea2-istiod-ca-cert" pod "82ef457f-b231-432d-9c89-52e22180cea2" (UID: "82ef457f-b231-432d-9c89-52e22180cea2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:07:59.644617 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.644570    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-istio-data\") pod \"82ef457f-b231-432d-9c89-52e22180cea2\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") "
Nov 03 21:07:59.644807 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.644619    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-workload-socket\") pod \"82ef457f-b231-432d-9c89-52e22180cea2\" (UID: \"82ef457f-b231-432d-9c89-52e22180cea2\") "
Nov 03 21:07:59.644942 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.644908    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-credential-socket" pod "82ef457f-b231-432d-9c89-52e22180cea2" (UID: "82ef457f-b231-432d-9c89-52e22180cea2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:59.644942 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.644925    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-istio-data" pod "82ef457f-b231-432d-9c89-52e22180cea2" (UID: "82ef457f-b231-432d-9c89-52e22180cea2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:59.645111 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.644993    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-workload-socket" pod "82ef457f-b231-432d-9c89-52e22180cea2" (UID: "82ef457f-b231-432d-9c89-52e22180cea2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:59.645538 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.645280    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:59.645538 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.645389    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:59.645538 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.645485    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82ef457f-b231-432d-9c89-52e22180cea2-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:59.645538 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.645499    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:59.645538 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.645514    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:59.647081 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.647054    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82ef457f-b231-432d-9c89-52e22180cea2-istio-token" pod "82ef457f-b231-432d-9c89-52e22180cea2" (UID: "82ef457f-b231-432d-9c89-52e22180cea2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:59.647349 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.647294    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82ef457f-b231-432d-9c89-52e22180cea2-kube-api-access-4mpck" pod "82ef457f-b231-432d-9c89-52e22180cea2" (UID: "82ef457f-b231-432d-9c89-52e22180cea2"). InnerVolumeSpecName "kube-api-access-4mpck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:07:59.647427 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.647375    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-istio-envoy" pod "82ef457f-b231-432d-9c89-52e22180cea2" (UID: "82ef457f-b231-432d-9c89-52e22180cea2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:07:59.648880 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.648858    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/82ef457f-b231-432d-9c89-52e22180cea2-istio-podinfo" pod "82ef457f-b231-432d-9c89-52e22180cea2" (UID: "82ef457f-b231-432d-9c89-52e22180cea2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:07:59.685228 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.685199    4993 generic.go:356] "Generic (PLEG): container finished" podID="82ef457f-b231-432d-9c89-52e22180cea2" containerID="ea0bc4938dc734c4343b185809cb4d48f2081f67e46034d706b67b03b9fa6026" exitCode=0
Nov 03 21:07:59.685343 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.685274    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr" event={"ID":"82ef457f-b231-432d-9c89-52e22180cea2","Type":"ContainerDied","Data":"ea0bc4938dc734c4343b185809cb4d48f2081f67e46034d706b67b03b9fa6026"}
Nov 03 21:07:59.685343 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.685307    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"
Nov 03 21:07:59.685343 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.685321    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr" event={"ID":"82ef457f-b231-432d-9c89-52e22180cea2","Type":"ContainerDied","Data":"698dde6abaf066c504f6dc0844ac962e77fbed0bcbee67c9beb51e052b3a06f7"}
Nov 03 21:07:59.685343 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.685343    4993 scope.go:122] "RemoveContainer" containerID="ea0bc4938dc734c4343b185809cb4d48f2081f67e46034d706b67b03b9fa6026"
Nov 03 21:07:59.696766 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.696748    4993 scope.go:122] "RemoveContainer" containerID="ea0bc4938dc734c4343b185809cb4d48f2081f67e46034d706b67b03b9fa6026"
Nov 03 21:07:59.697074 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:07:59.697051    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea0bc4938dc734c4343b185809cb4d48f2081f67e46034d706b67b03b9fa6026\": container with ID starting with ea0bc4938dc734c4343b185809cb4d48f2081f67e46034d706b67b03b9fa6026 not found: ID does not exist" containerID="ea0bc4938dc734c4343b185809cb4d48f2081f67e46034d706b67b03b9fa6026"
Nov 03 21:07:59.697125 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.697083    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea0bc4938dc734c4343b185809cb4d48f2081f67e46034d706b67b03b9fa6026"} err="failed to get container status \"ea0bc4938dc734c4343b185809cb4d48f2081f67e46034d706b67b03b9fa6026\": rpc error: code = NotFound desc = could not find container \"ea0bc4938dc734c4343b185809cb4d48f2081f67e46034d706b67b03b9fa6026\": container with ID starting with ea0bc4938dc734c4343b185809cb4d48f2081f67e46034d706b67b03b9fa6026 not found: ID does not exist"
Nov 03 21:07:59.747844 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.747817    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82ef457f-b231-432d-9c89-52e22180cea2-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:59.747844 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.747841    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82ef457f-b231-432d-9c89-52e22180cea2-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:59.747844 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.747851    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82ef457f-b231-432d-9c89-52e22180cea2-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:59.748073 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.747862    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mpck\" (UniqueName: \"kubernetes.io/projected/82ef457f-b231-432d-9c89-52e22180cea2-kube-api-access-4mpck\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:07:59.835465 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.835375    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"]
Nov 03 21:07:59.842267 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:07:59.842240    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-2bksr"]
Nov 03 21:08:01.579791 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:01.579757    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82ef457f-b231-432d-9c89-52e22180cea2" path="/var/lib/kubelet/pods/82ef457f-b231-432d-9c89-52e22180cea2/volumes"
Nov 03 21:08:01.670280 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:08:01.670245    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:08:23.435315 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:23.435279    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"]
Nov 03 21:08:23.484444 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:23.484413    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"]
Nov 03 21:08:31.735247 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:08:31.735206    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:08:38.457129 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:38.457029    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main" containerID="cri-o://85c3797366f9f15f0b7bb5ee2452dc82578ca3f8e061878e3dc9bc6f08ad0be3" gracePeriod=45
Nov 03 21:08:38.513999 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:38.513808    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" containerID="cri-o://c8f1f1b552efd36d52e1a51a53bb21d7267eb7edd1d916b6c1ef27aa088d4575" gracePeriod=45
Nov 03 21:08:38.555917 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:38.555842    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": EOF"
Nov 03 21:08:39.052045 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.046719    4993 generic.go:356] "Generic (PLEG): container finished" podID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerID="85c3797366f9f15f0b7bb5ee2452dc82578ca3f8e061878e3dc9bc6f08ad0be3" exitCode=0
Nov 03 21:08:39.052045 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.046920    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" event={"ID":"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e","Type":"ContainerDied","Data":"85c3797366f9f15f0b7bb5ee2452dc82578ca3f8e061878e3dc9bc6f08ad0be3"}
Nov 03 21:08:39.052045 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.049244    4993 generic.go:356] "Generic (PLEG): container finished" podID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerID="c8f1f1b552efd36d52e1a51a53bb21d7267eb7edd1d916b6c1ef27aa088d4575" exitCode=0
Nov 03 21:08:39.052045 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.049348    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" event={"ID":"38eea1db-b2e8-4364-a7a9-b7bb655a20f1","Type":"ContainerDied","Data":"c8f1f1b552efd36d52e1a51a53bb21d7267eb7edd1d916b6c1ef27aa088d4575"}
Nov 03 21:08:39.209704 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.202251    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 21:08:39.257680 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.257006    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-home\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-home\") pod \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") "
Nov 03 21:08:39.257680 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.257105    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-tmp-dir\") pod \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") "
Nov 03 21:08:39.257680 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.257153    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-dshm\") pod \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") "
Nov 03 21:08:39.257680 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.257188    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-model-cache\") pod \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") "
Nov 03 21:08:39.257680 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.257252    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-tls-certs\" (UniqueName: \"kubernetes.io/secret/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-tls-certs\") pod \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") "
Nov 03 21:08:39.257680 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.257397    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-kube-api-access-q7nqm\" (UniqueName: \"kubernetes.io/projected/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-kube-api-access-q7nqm\") pod \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\" (UID: \"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e\") "
Nov 03 21:08:39.259608 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.259315    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-home" pod "bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" (UID: "bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:39.259608 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.259328    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-tmp-dir" pod "bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" (UID: "bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:39.259833 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.259693    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-model-cache" pod "bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" (UID: "bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:39.261718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.261690    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-kube-api-access-q7nqm" pod "bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" (UID: "bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e"). InnerVolumeSpecName "kube-api-access-q7nqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:39.263095 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.263061    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-dshm" pod "bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" (UID: "bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:39.314609 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.314522    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-tls-certs" pod "bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" (UID: "bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:08:39.359898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.359568    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7nqm\" (UniqueName: \"kubernetes.io/projected/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-kube-api-access-q7nqm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:39.359898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.359607    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:39.359898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.359623    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:39.359898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.359655    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:39.359898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.359668    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:39.359898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.359682    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:39.693863 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.693836    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 21:08:39.802091 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.802055    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-home\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-home\") pod \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") "
Nov 03 21:08:39.802282 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.802106    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-dshm\") pod \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") "
Nov 03 21:08:39.802282 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.802146    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-kube-api-access-49bgb\" (UniqueName: \"kubernetes.io/projected/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-kube-api-access-49bgb\") pod \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") "
Nov 03 21:08:39.802282 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.802192    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-tls-certs\" (UniqueName: \"kubernetes.io/secret/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-tls-certs\") pod \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") "
Nov 03 21:08:39.802282 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.802259    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-tmp-dir\") pod \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") "
Nov 03 21:08:39.802506 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.802335    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-model-cache\") pod \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\" (UID: \"38eea1db-b2e8-4364-a7a9-b7bb655a20f1\") "
Nov 03 21:08:39.802506 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.802368    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-home" pod "38eea1db-b2e8-4364-a7a9-b7bb655a20f1" (UID: "38eea1db-b2e8-4364-a7a9-b7bb655a20f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:39.803268 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.802921    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-model-cache" pod "38eea1db-b2e8-4364-a7a9-b7bb655a20f1" (UID: "38eea1db-b2e8-4364-a7a9-b7bb655a20f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:39.803268 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.803020    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-tmp-dir" pod "38eea1db-b2e8-4364-a7a9-b7bb655a20f1" (UID: "38eea1db-b2e8-4364-a7a9-b7bb655a20f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:39.803566 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.803484    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:39.803566 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.803524    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:39.803566 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.803542    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:39.805372 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.805039    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-dshm" pod "38eea1db-b2e8-4364-a7a9-b7bb655a20f1" (UID: "38eea1db-b2e8-4364-a7a9-b7bb655a20f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:08:39.805749 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.805716    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-tls-certs" pod "38eea1db-b2e8-4364-a7a9-b7bb655a20f1" (UID: "38eea1db-b2e8-4364-a7a9-b7bb655a20f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:08:39.806271 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.806253    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-kube-api-access-49bgb" pod "38eea1db-b2e8-4364-a7a9-b7bb655a20f1" (UID: "38eea1db-b2e8-4364-a7a9-b7bb655a20f1"). InnerVolumeSpecName "kube-api-access-49bgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:08:39.905452 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.905370    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:39.905452 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.905416    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49bgb\" (UniqueName: \"kubernetes.io/projected/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-kube-api-access-49bgb\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:39.905452 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:39.905434    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38eea1db-b2e8-4364-a7a9-b7bb655a20f1-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:08:40.059551 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:40.059512    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"
Nov 03 21:08:40.059551 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:40.059536    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" event={"ID":"38eea1db-b2e8-4364-a7a9-b7bb655a20f1","Type":"ContainerDied","Data":"622e19c0cc850813b2622703a027e091c3c90b61d2212c4551069e49b583da27"}
Nov 03 21:08:40.059827 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:40.059588    4993 scope.go:122] "RemoveContainer" containerID="c8f1f1b552efd36d52e1a51a53bb21d7267eb7edd1d916b6c1ef27aa088d4575"
Nov 03 21:08:40.061380 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:40.061351    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m" event={"ID":"bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e","Type":"ContainerDied","Data":"0177881eeb2d3f99fd467d4b771a377a55524991ffb85c44f753a448cb3c2385"}
Nov 03 21:08:40.061497 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:40.061452    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"
Nov 03 21:08:40.081545 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:40.081523    4993 scope.go:122] "RemoveContainer" containerID="85c3797366f9f15f0b7bb5ee2452dc82578ca3f8e061878e3dc9bc6f08ad0be3"
Nov 03 21:08:40.236173 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:40.234711    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"]
Nov 03 21:08:40.238457 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:40.238426    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-89bb58/distinct-2-w3-89bb58-kserve-c4cb8447-mmf2m"]
Nov 03 21:08:40.399652 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:40.399030    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"]
Nov 03 21:08:40.427263 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:40.427189    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6"]
Nov 03 21:08:40.486299 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:40.485722    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-89bb58/simulator-w3-89bb58-kserve-7fbdf9d748-vxjd6" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:08:41.585709 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:41.585674    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" path="/var/lib/kubelet/pods/38eea1db-b2e8-4364-a7a9-b7bb655a20f1/volumes"
Nov 03 21:08:41.587044 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:41.587018    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" path="/var/lib/kubelet/pods/bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e/volumes"
Nov 03 21:08:41.997655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:41.997079    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:08:41.997655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:08:41.997163    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p" podUID="6bf4e14e-9896-452a-bfe9-58e3f89a155c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:09:01.676314 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:09:01.676273    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:09:22.659955 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:22.659921    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"]
Nov 03 21:09:22.709930 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:22.709894    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"]
Nov 03 21:09:31.728164 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:09:31.728112    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:09:34.906945 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:34.906907    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"]
Nov 03 21:09:34.956868 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:34.956815    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"]
Nov 03 21:09:37.681275 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:37.681233    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main" containerID="cri-o://9d53d0b8a0ad644107144ce5babd9312a7ce533ee4d449a53fb25c595d6cbd7b" gracePeriod=45
Nov 03 21:09:37.731288 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:37.731251    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" containerID="cri-o://e5672705e69f44aa10e25023c7ef4a69c286ec738d49d99bd6b0b441d00fb30f" gracePeriod=45
Nov 03 21:09:38.045840 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.045543    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 21:09:38.051059 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.051027    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 21:09:38.129715 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.129688    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-kube-api-access-cdzwt\" (UniqueName: \"kubernetes.io/projected/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-kube-api-access-cdzwt\") pod \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") "
Nov 03 21:09:38.129892 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.129746    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-model-cache\") pod \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") "
Nov 03 21:09:38.129892 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.129804    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-tmp-dir\") pod \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") "
Nov 03 21:09:38.129892 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.129826    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-tls-certs\" (UniqueName: \"kubernetes.io/secret/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-tls-certs\") pod \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") "
Nov 03 21:09:38.129892 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.129855    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-dshm\") pod \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") "
Nov 03 21:09:38.130123 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.129909    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-tmp-dir\") pod \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") "
Nov 03 21:09:38.130123 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.129937    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-kube-api-access-525tq\" (UniqueName: \"kubernetes.io/projected/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-kube-api-access-525tq\") pod \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") "
Nov 03 21:09:38.130123 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130014    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-tls-certs\" (UniqueName: \"kubernetes.io/secret/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-tls-certs\") pod \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") "
Nov 03 21:09:38.130123 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130041    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-home\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-home\") pod \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") "
Nov 03 21:09:38.130123 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130074    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-dshm\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-dshm\") pod \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\" (UID: \"933b980b-58f5-44d0-b7b6-9f3fd58bcb63\") "
Nov 03 21:09:38.130386 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130118    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-model-cache" pod "933b980b-58f5-44d0-b7b6-9f3fd58bcb63" (UID: "933b980b-58f5-44d0-b7b6-9f3fd58bcb63"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:38.130386 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130108    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-tmp-dir" pod "933b980b-58f5-44d0-b7b6-9f3fd58bcb63" (UID: "933b980b-58f5-44d0-b7b6-9f3fd58bcb63"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:38.130386 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130148    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-model-cache\") pod \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") "
Nov 03 21:09:38.130386 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130219    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-home\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-home\") pod \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\" (UID: \"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e\") "
Nov 03 21:09:38.130386 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130348    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-tmp-dir" pod "c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" (UID: "c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:38.130386 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130352    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-home" pod "933b980b-58f5-44d0-b7b6-9f3fd58bcb63" (UID: "933b980b-58f5-44d0-b7b6-9f3fd58bcb63"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:38.130727 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130564    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-model-cache" pod "c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" (UID: "c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:38.130937 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130879    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:38.131018 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130944    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:38.131018 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130944    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-home" pod "c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" (UID: "c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:38.131018 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130959    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:38.131018 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130971    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:38.131018 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.130983    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:38.132880 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.132859    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-tls-certs" pod "933b980b-58f5-44d0-b7b6-9f3fd58bcb63" (UID: "933b980b-58f5-44d0-b7b6-9f3fd58bcb63"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:09:38.133014 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.132958    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-kube-api-access-cdzwt" pod "933b980b-58f5-44d0-b7b6-9f3fd58bcb63" (UID: "933b980b-58f5-44d0-b7b6-9f3fd58bcb63"). InnerVolumeSpecName "kube-api-access-cdzwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:09:38.133187 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.133166    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-kube-api-access-525tq" pod "c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" (UID: "c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e"). InnerVolumeSpecName "kube-api-access-525tq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:09:38.133316 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.133287    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-dshm" pod "933b980b-58f5-44d0-b7b6-9f3fd58bcb63" (UID: "933b980b-58f5-44d0-b7b6-9f3fd58bcb63"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:38.133449 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.133430    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-tls-certs" pod "c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" (UID: "c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:09:38.133556 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.133466    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-dshm" pod "c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" (UID: "c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:38.232534 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.232500    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:38.232534 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.232534    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:38.232781 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.232550    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-525tq\" (UniqueName: \"kubernetes.io/projected/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-kube-api-access-525tq\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:38.232781 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.232561    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:38.232781 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.232570    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:38.232781 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.232577    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:38.232781 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.232585    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdzwt\" (UniqueName: \"kubernetes.io/projected/933b980b-58f5-44d0-b7b6-9f3fd58bcb63-kube-api-access-cdzwt\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:38.589586 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.589542    4993 generic.go:356] "Generic (PLEG): container finished" podID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerID="9d53d0b8a0ad644107144ce5babd9312a7ce533ee4d449a53fb25c595d6cbd7b" exitCode=0
Nov 03 21:09:38.589814 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.589587    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"
Nov 03 21:09:38.589814 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.589613    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" event={"ID":"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e","Type":"ContainerDied","Data":"9d53d0b8a0ad644107144ce5babd9312a7ce533ee4d449a53fb25c595d6cbd7b"}
Nov 03 21:09:38.589814 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.589695    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s" event={"ID":"c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e","Type":"ContainerDied","Data":"2167912ed5c2de0bb62c1ec3754005900b428ea0eb821876614de5d875176fa0"}
Nov 03 21:09:38.589814 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.589715    4993 scope.go:122] "RemoveContainer" containerID="9d53d0b8a0ad644107144ce5babd9312a7ce533ee4d449a53fb25c595d6cbd7b"
Nov 03 21:09:38.591105 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.591081    4993 generic.go:356] "Generic (PLEG): container finished" podID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerID="e5672705e69f44aa10e25023c7ef4a69c286ec738d49d99bd6b0b441d00fb30f" exitCode=0
Nov 03 21:09:38.591201 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.591185    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" event={"ID":"933b980b-58f5-44d0-b7b6-9f3fd58bcb63","Type":"ContainerDied","Data":"e5672705e69f44aa10e25023c7ef4a69c286ec738d49d99bd6b0b441d00fb30f"}
Nov 03 21:09:38.591254 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.591219    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" event={"ID":"933b980b-58f5-44d0-b7b6-9f3fd58bcb63","Type":"ContainerDied","Data":"9adf7ce9a3bf80d4a2b76c4797a24463a47de8923e61287e4c459b1c7cfc3943"}
Nov 03 21:09:38.591290 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.591278    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"
Nov 03 21:09:38.609895 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.609859    4993 scope.go:122] "RemoveContainer" containerID="9d53d0b8a0ad644107144ce5babd9312a7ce533ee4d449a53fb25c595d6cbd7b"
Nov 03 21:09:38.610374 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:09:38.610346    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d53d0b8a0ad644107144ce5babd9312a7ce533ee4d449a53fb25c595d6cbd7b\": container with ID starting with 9d53d0b8a0ad644107144ce5babd9312a7ce533ee4d449a53fb25c595d6cbd7b not found: ID does not exist" containerID="9d53d0b8a0ad644107144ce5babd9312a7ce533ee4d449a53fb25c595d6cbd7b"
Nov 03 21:09:38.610490 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.610381    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d53d0b8a0ad644107144ce5babd9312a7ce533ee4d449a53fb25c595d6cbd7b"} err="failed to get container status \"9d53d0b8a0ad644107144ce5babd9312a7ce533ee4d449a53fb25c595d6cbd7b\": rpc error: code = NotFound desc = could not find container \"9d53d0b8a0ad644107144ce5babd9312a7ce533ee4d449a53fb25c595d6cbd7b\": container with ID starting with 9d53d0b8a0ad644107144ce5babd9312a7ce533ee4d449a53fb25c595d6cbd7b not found: ID does not exist"
Nov 03 21:09:38.610490 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.610407    4993 scope.go:122] "RemoveContainer" containerID="e5672705e69f44aa10e25023c7ef4a69c286ec738d49d99bd6b0b441d00fb30f"
Nov 03 21:09:38.620934 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.620916    4993 scope.go:122] "RemoveContainer" containerID="e5672705e69f44aa10e25023c7ef4a69c286ec738d49d99bd6b0b441d00fb30f"
Nov 03 21:09:38.621186 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:09:38.621164    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5672705e69f44aa10e25023c7ef4a69c286ec738d49d99bd6b0b441d00fb30f\": container with ID starting with e5672705e69f44aa10e25023c7ef4a69c286ec738d49d99bd6b0b441d00fb30f not found: ID does not exist" containerID="e5672705e69f44aa10e25023c7ef4a69c286ec738d49d99bd6b0b441d00fb30f"
Nov 03 21:09:38.621232 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.621196    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5672705e69f44aa10e25023c7ef4a69c286ec738d49d99bd6b0b441d00fb30f"} err="failed to get container status \"e5672705e69f44aa10e25023c7ef4a69c286ec738d49d99bd6b0b441d00fb30f\": rpc error: code = NotFound desc = could not find container \"e5672705e69f44aa10e25023c7ef4a69c286ec738d49d99bd6b0b441d00fb30f\": container with ID starting with e5672705e69f44aa10e25023c7ef4a69c286ec738d49d99bd6b0b441d00fb30f not found: ID does not exist"
Nov 03 21:09:38.759601 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.759569    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"]
Nov 03 21:09:38.808969 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.808935    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969"]
Nov 03 21:09:38.915616 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.915531    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"]
Nov 03 21:09:38.922576 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.922547    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-ca4642/distinct-w4-ca4642-kserve-68fc776df-z5k2s"]
Nov 03 21:09:38.941029 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:38.940996    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-ca4642/premium-w4-ca4642-kserve-57b744747d-s9969" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main" probeResult="failure" output="Get \"https://10.133.0.44:8000/ready\": context deadline exceeded"
Nov 03 21:09:39.577975 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:39.577944    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" path="/var/lib/kubelet/pods/933b980b-58f5-44d0-b7b6-9f3fd58bcb63/volumes"
Nov 03 21:09:39.578390 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:39.578377    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" path="/var/lib/kubelet/pods/c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e/volumes"
Nov 03 21:09:49.950653 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:49.950187    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main" containerID="cri-o://0481e49b495dbd704128c4836ce10dac0842da047b1820ddfebff8fd85370338" gracePeriod=45
Nov 03 21:09:50.008654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.005821    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" containerID="cri-o://56273e48a1c233067e26ba0d5a570a6f67b14a6cd52f8987b1936de2e7bdff68" gracePeriod=45
Nov 03 21:09:50.267338 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.267305    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:09:50.448535 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.446132    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-model-cache\") pod \"5af5bf28-fe61-4653-b16a-8640b3156119\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") "
Nov 03 21:09:50.448535 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.446217    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5af5bf28-fe61-4653-b16a-8640b3156119-tls-certs\" (UniqueName: \"kubernetes.io/secret/5af5bf28-fe61-4653-b16a-8640b3156119-tls-certs\") pod \"5af5bf28-fe61-4653-b16a-8640b3156119\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") "
Nov 03 21:09:50.448535 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.446333    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-tmp-dir\") pod \"5af5bf28-fe61-4653-b16a-8640b3156119\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") "
Nov 03 21:09:50.448535 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.446362    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5af5bf28-fe61-4653-b16a-8640b3156119-kube-api-access-9qdkm\" (UniqueName: \"kubernetes.io/projected/5af5bf28-fe61-4653-b16a-8640b3156119-kube-api-access-9qdkm\") pod \"5af5bf28-fe61-4653-b16a-8640b3156119\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") "
Nov 03 21:09:50.448535 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.446436    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-home\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-home\") pod \"5af5bf28-fe61-4653-b16a-8640b3156119\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") "
Nov 03 21:09:50.448535 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.446499    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-dshm\") pod \"5af5bf28-fe61-4653-b16a-8640b3156119\" (UID: \"5af5bf28-fe61-4653-b16a-8640b3156119\") "
Nov 03 21:09:50.448535 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.447429    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-home" pod "5af5bf28-fe61-4653-b16a-8640b3156119" (UID: "5af5bf28-fe61-4653-b16a-8640b3156119"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:50.448535 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.447941    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-model-cache" pod "5af5bf28-fe61-4653-b16a-8640b3156119" (UID: "5af5bf28-fe61-4653-b16a-8640b3156119"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:50.448535 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.448236    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-tmp-dir" pod "5af5bf28-fe61-4653-b16a-8640b3156119" (UID: "5af5bf28-fe61-4653-b16a-8640b3156119"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:50.449504 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.449435    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:50.449504 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.449466    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:50.449504 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.449481    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:50.457841 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.457805    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5af5bf28-fe61-4653-b16a-8640b3156119-tls-certs" pod "5af5bf28-fe61-4653-b16a-8640b3156119" (UID: "5af5bf28-fe61-4653-b16a-8640b3156119"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:09:50.458334 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.458191    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-dshm" pod "5af5bf28-fe61-4653-b16a-8640b3156119" (UID: "5af5bf28-fe61-4653-b16a-8640b3156119"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:50.475010 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.474977    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5af5bf28-fe61-4653-b16a-8640b3156119-kube-api-access-9qdkm" pod "5af5bf28-fe61-4653-b16a-8640b3156119" (UID: "5af5bf28-fe61-4653-b16a-8640b3156119"). InnerVolumeSpecName "kube-api-access-9qdkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:09:50.480613 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.480555    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 21:09:50.551189 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.551158    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5af5bf28-fe61-4653-b16a-8640b3156119-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:50.551189 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.551184    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qdkm\" (UniqueName: \"kubernetes.io/projected/5af5bf28-fe61-4653-b16a-8640b3156119-kube-api-access-9qdkm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:50.551399 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.551199    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5af5bf28-fe61-4653-b16a-8640b3156119-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:50.651995 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.651950    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-home\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-home\") pod \"df7bec60-2e19-4184-9432-a08efd6ea203\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") "
Nov 03 21:09:50.652180 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.652053    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df7bec60-2e19-4184-9432-a08efd6ea203-kube-api-access-v84hq\" (UniqueName: \"kubernetes.io/projected/df7bec60-2e19-4184-9432-a08efd6ea203-kube-api-access-v84hq\") pod \"df7bec60-2e19-4184-9432-a08efd6ea203\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") "
Nov 03 21:09:50.652180 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.652086    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df7bec60-2e19-4184-9432-a08efd6ea203-tls-certs\" (UniqueName: \"kubernetes.io/secret/df7bec60-2e19-4184-9432-a08efd6ea203-tls-certs\") pod \"df7bec60-2e19-4184-9432-a08efd6ea203\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") "
Nov 03 21:09:50.652180 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.652145    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-dshm\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-dshm\") pod \"df7bec60-2e19-4184-9432-a08efd6ea203\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") "
Nov 03 21:09:50.652371 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.652208    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-model-cache\") pod \"df7bec60-2e19-4184-9432-a08efd6ea203\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") "
Nov 03 21:09:50.652371 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.652265    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-tmp-dir\") pod \"df7bec60-2e19-4184-9432-a08efd6ea203\" (UID: \"df7bec60-2e19-4184-9432-a08efd6ea203\") "
Nov 03 21:09:50.652854 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.652652    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-home" pod "df7bec60-2e19-4184-9432-a08efd6ea203" (UID: "df7bec60-2e19-4184-9432-a08efd6ea203"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:50.652974 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.652864    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-tmp-dir" pod "df7bec60-2e19-4184-9432-a08efd6ea203" (UID: "df7bec60-2e19-4184-9432-a08efd6ea203"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:50.653541 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.653496    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-model-cache" pod "df7bec60-2e19-4184-9432-a08efd6ea203" (UID: "df7bec60-2e19-4184-9432-a08efd6ea203"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:50.655047 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.655026    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df7bec60-2e19-4184-9432-a08efd6ea203-tls-certs" pod "df7bec60-2e19-4184-9432-a08efd6ea203" (UID: "df7bec60-2e19-4184-9432-a08efd6ea203"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:09:50.655351 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.655315    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df7bec60-2e19-4184-9432-a08efd6ea203-kube-api-access-v84hq" pod "df7bec60-2e19-4184-9432-a08efd6ea203" (UID: "df7bec60-2e19-4184-9432-a08efd6ea203"). InnerVolumeSpecName "kube-api-access-v84hq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:09:50.656037 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.656012    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-dshm" pod "df7bec60-2e19-4184-9432-a08efd6ea203" (UID: "df7bec60-2e19-4184-9432-a08efd6ea203"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:09:50.703499 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.703459    4993 generic.go:356] "Generic (PLEG): container finished" podID="5af5bf28-fe61-4653-b16a-8640b3156119" containerID="0481e49b495dbd704128c4836ce10dac0842da047b1820ddfebff8fd85370338" exitCode=0
Nov 03 21:09:50.703750 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.703667    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" event={"ID":"5af5bf28-fe61-4653-b16a-8640b3156119","Type":"ContainerDied","Data":"0481e49b495dbd704128c4836ce10dac0842da047b1820ddfebff8fd85370338"}
Nov 03 21:09:50.703750 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.703710    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b" event={"ID":"5af5bf28-fe61-4653-b16a-8640b3156119","Type":"ContainerDied","Data":"73ada115ed9f98ef42b8c85a41aa16e7714948c3dc7725fc12bfd7f32db9449b"}
Nov 03 21:09:50.703750 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.703732    4993 scope.go:122] "RemoveContainer" containerID="0481e49b495dbd704128c4836ce10dac0842da047b1820ddfebff8fd85370338"
Nov 03 21:09:50.703904 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.703889    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"
Nov 03 21:09:50.718721 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.713359    4993 generic.go:356] "Generic (PLEG): container finished" podID="df7bec60-2e19-4184-9432-a08efd6ea203" containerID="56273e48a1c233067e26ba0d5a570a6f67b14a6cd52f8987b1936de2e7bdff68" exitCode=0
Nov 03 21:09:50.718721 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.713445    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" event={"ID":"df7bec60-2e19-4184-9432-a08efd6ea203","Type":"ContainerDied","Data":"56273e48a1c233067e26ba0d5a570a6f67b14a6cd52f8987b1936de2e7bdff68"}
Nov 03 21:09:50.718721 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.713476    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" event={"ID":"df7bec60-2e19-4184-9432-a08efd6ea203","Type":"ContainerDied","Data":"8aa81e4a223bee7d47e889297a3c75845b1af9a9ca736bb50702efb557d6326d"}
Nov 03 21:09:50.718721 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.713540    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"
Nov 03 21:09:50.726658 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.726590    4993 scope.go:122] "RemoveContainer" containerID="0481e49b495dbd704128c4836ce10dac0842da047b1820ddfebff8fd85370338"
Nov 03 21:09:50.728579 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:09:50.727335    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0481e49b495dbd704128c4836ce10dac0842da047b1820ddfebff8fd85370338\": container with ID starting with 0481e49b495dbd704128c4836ce10dac0842da047b1820ddfebff8fd85370338 not found: ID does not exist" containerID="0481e49b495dbd704128c4836ce10dac0842da047b1820ddfebff8fd85370338"
Nov 03 21:09:50.730791 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.730695    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0481e49b495dbd704128c4836ce10dac0842da047b1820ddfebff8fd85370338"} err="failed to get container status \"0481e49b495dbd704128c4836ce10dac0842da047b1820ddfebff8fd85370338\": rpc error: code = NotFound desc = could not find container \"0481e49b495dbd704128c4836ce10dac0842da047b1820ddfebff8fd85370338\": container with ID starting with 0481e49b495dbd704128c4836ce10dac0842da047b1820ddfebff8fd85370338 not found: ID does not exist"
Nov 03 21:09:50.730791 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.730763    4993 scope.go:122] "RemoveContainer" containerID="56273e48a1c233067e26ba0d5a570a6f67b14a6cd52f8987b1936de2e7bdff68"
Nov 03 21:09:50.740320 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.740300    4993 scope.go:122] "RemoveContainer" containerID="56273e48a1c233067e26ba0d5a570a6f67b14a6cd52f8987b1936de2e7bdff68"
Nov 03 21:09:50.740597 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:09:50.740577    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56273e48a1c233067e26ba0d5a570a6f67b14a6cd52f8987b1936de2e7bdff68\": container with ID starting with 56273e48a1c233067e26ba0d5a570a6f67b14a6cd52f8987b1936de2e7bdff68 not found: ID does not exist" containerID="56273e48a1c233067e26ba0d5a570a6f67b14a6cd52f8987b1936de2e7bdff68"
Nov 03 21:09:50.740649 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.740605    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56273e48a1c233067e26ba0d5a570a6f67b14a6cd52f8987b1936de2e7bdff68"} err="failed to get container status \"56273e48a1c233067e26ba0d5a570a6f67b14a6cd52f8987b1936de2e7bdff68\": rpc error: code = NotFound desc = could not find container \"56273e48a1c233067e26ba0d5a570a6f67b14a6cd52f8987b1936de2e7bdff68\": container with ID starting with 56273e48a1c233067e26ba0d5a570a6f67b14a6cd52f8987b1936de2e7bdff68 not found: ID does not exist"
Nov 03 21:09:50.755242 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.755220    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:50.755356 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.755245    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:50.755356 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.755257    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:50.755356 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.755269    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v84hq\" (UniqueName: \"kubernetes.io/projected/df7bec60-2e19-4184-9432-a08efd6ea203-kube-api-access-v84hq\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:50.755356 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.755283    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df7bec60-2e19-4184-9432-a08efd6ea203-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:50.755356 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.755294    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df7bec60-2e19-4184-9432-a08efd6ea203-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:09:50.904364 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.904295    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"]
Nov 03 21:09:50.938868 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:50.938797    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x"]
Nov 03 21:09:51.004986 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:51.004899    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"]
Nov 03 21:09:51.055125 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:51.055094    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-499187/distinct-2-w0-499187-kserve-56fd64468b-w9m8b"]
Nov 03 21:09:51.451386 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:51.451292    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-499187/premium-w0-499187-kserve-7774c96bc4-5k27x" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main" probeResult="failure" output="Get \"https://10.133.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:09:51.577816 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:51.577786    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" path="/var/lib/kubelet/pods/5af5bf28-fe61-4653-b16a-8640b3156119/volumes"
Nov 03 21:09:51.578224 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:51.578210    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" path="/var/lib/kubelet/pods/df7bec60-2e19-4184-9432-a08efd6ea203/volumes"
Nov 03 21:09:59.960237 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:59.960200    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"]
Nov 03 21:09:59.960662 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:09:59.960427    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv" podUID="b380cf76-c519-4719-84fa-5c3025ea84f1" containerName="maas-api" containerID="cri-o://6f1b2bcab3ed951a0e54adc78b2c60cac813eeab3f70773a0a0cc0b01be902e4" gracePeriod=30
Nov 03 21:10:00.335483 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.335457    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 21:10:00.435655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.433072    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b380cf76-c519-4719-84fa-5c3025ea84f1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b380cf76-c519-4719-84fa-5c3025ea84f1-openshift-service-ca\") pod \"b380cf76-c519-4719-84fa-5c3025ea84f1\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") "
Nov 03 21:10:00.435655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.433206    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-maas-api-tls\") pod \"b380cf76-c519-4719-84fa-5c3025ea84f1\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") "
Nov 03 21:10:00.435655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.433255    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b380cf76-c519-4719-84fa-5c3025ea84f1-kube-api-access-f7bsz\" (UniqueName: \"kubernetes.io/projected/b380cf76-c519-4719-84fa-5c3025ea84f1-kube-api-access-f7bsz\") pod \"b380cf76-c519-4719-84fa-5c3025ea84f1\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") "
Nov 03 21:10:00.435655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.433304    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-metrics-certs\") pod \"b380cf76-c519-4719-84fa-5c3025ea84f1\" (UID: \"b380cf76-c519-4719-84fa-5c3025ea84f1\") "
Nov 03 21:10:00.435655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.435337    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b380cf76-c519-4719-84fa-5c3025ea84f1-openshift-service-ca" pod "b380cf76-c519-4719-84fa-5c3025ea84f1" (UID: "b380cf76-c519-4719-84fa-5c3025ea84f1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:10:00.448654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.447820    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b380cf76-c519-4719-84fa-5c3025ea84f1-kube-api-access-f7bsz" pod "b380cf76-c519-4719-84fa-5c3025ea84f1" (UID: "b380cf76-c519-4719-84fa-5c3025ea84f1"). InnerVolumeSpecName "kube-api-access-f7bsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:10:00.448839 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.448682    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-metrics-certs" pod "b380cf76-c519-4719-84fa-5c3025ea84f1" (UID: "b380cf76-c519-4719-84fa-5c3025ea84f1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:10:00.448839 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.448735    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-maas-api-tls" pod "b380cf76-c519-4719-84fa-5c3025ea84f1" (UID: "b380cf76-c519-4719-84fa-5c3025ea84f1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:10:00.540285 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.540248    4993 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b380cf76-c519-4719-84fa-5c3025ea84f1-openshift-service-ca\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:00.540517 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.540500    4993 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-maas-api-tls\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:00.540607 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.540596    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7bsz\" (UniqueName: \"kubernetes.io/projected/b380cf76-c519-4719-84fa-5c3025ea84f1-kube-api-access-f7bsz\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:00.540748 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.540735    4993 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b380cf76-c519-4719-84fa-5c3025ea84f1-metrics-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:00.779215 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.779175    4993 generic.go:356] "Generic (PLEG): container finished" podID="b380cf76-c519-4719-84fa-5c3025ea84f1" containerID="6f1b2bcab3ed951a0e54adc78b2c60cac813eeab3f70773a0a0cc0b01be902e4" exitCode=0
Nov 03 21:10:00.779432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.779231    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"
Nov 03 21:10:00.779432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.779268    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv" event={"ID":"b380cf76-c519-4719-84fa-5c3025ea84f1","Type":"ContainerDied","Data":"6f1b2bcab3ed951a0e54adc78b2c60cac813eeab3f70773a0a0cc0b01be902e4"}
Nov 03 21:10:00.779432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.779330    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv" event={"ID":"b380cf76-c519-4719-84fa-5c3025ea84f1","Type":"ContainerDied","Data":"065160dfec5005be4862388905a78cabc45d96903ec509c0bd92275296fd7f24"}
Nov 03 21:10:00.779432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.779356    4993 scope.go:122] "RemoveContainer" containerID="6f1b2bcab3ed951a0e54adc78b2c60cac813eeab3f70773a0a0cc0b01be902e4"
Nov 03 21:10:00.791754 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.791738    4993 scope.go:122] "RemoveContainer" containerID="6f1b2bcab3ed951a0e54adc78b2c60cac813eeab3f70773a0a0cc0b01be902e4"
Nov 03 21:10:00.792067 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:10:00.792049    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f1b2bcab3ed951a0e54adc78b2c60cac813eeab3f70773a0a0cc0b01be902e4\": container with ID starting with 6f1b2bcab3ed951a0e54adc78b2c60cac813eeab3f70773a0a0cc0b01be902e4 not found: ID does not exist" containerID="6f1b2bcab3ed951a0e54adc78b2c60cac813eeab3f70773a0a0cc0b01be902e4"
Nov 03 21:10:00.792158 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.792074    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f1b2bcab3ed951a0e54adc78b2c60cac813eeab3f70773a0a0cc0b01be902e4"} err="failed to get container status \"6f1b2bcab3ed951a0e54adc78b2c60cac813eeab3f70773a0a0cc0b01be902e4\": rpc error: code = NotFound desc = could not find container \"6f1b2bcab3ed951a0e54adc78b2c60cac813eeab3f70773a0a0cc0b01be902e4\": container with ID starting with 6f1b2bcab3ed951a0e54adc78b2c60cac813eeab3f70773a0a0cc0b01be902e4 not found: ID does not exist"
Nov 03 21:10:00.946402 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:00.946355    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"]
Nov 03 21:10:01.033720 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:01.032915    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-89bb58-5675d949c6-8bshv"]
Nov 03 21:10:01.579069 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:01.579036    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b380cf76-c519-4719-84fa-5c3025ea84f1" path="/var/lib/kubelet/pods/b380cf76-c519-4719-84fa-5c3025ea84f1/volumes"
Nov 03 21:10:01.673387 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:10:01.673352    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:10:10.046397 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:10.046316    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"]
Nov 03 21:10:10.046798 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:10.046518    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48" podUID="d549e05f-a5a1-478c-bdb7-18b03af91b72" containerName="istio-proxy" containerID="cri-o://61f101a971bbe6978267554a0f431161bb6d8c00c6d16cb9e0583a08f1cb40d6" gracePeriod=30
Nov 03 21:10:10.065239 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:10.065207    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"]
Nov 03 21:10:10.065490 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:10.065450    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t" podUID="298f7d43-f4d7-4108-b8a1-8df13e36ccc9" containerName="istio-proxy" containerID="cri-o://f3f07d04f2f506a38a084245252bb375517ec0dc3ab33de427e1622338a1fb66" gracePeriod=30
Nov 03 21:10:10.068233 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:10.068204    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"]
Nov 03 21:10:10.068552 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:10.068435    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r" podUID="122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" containerName="istio-proxy" containerID="cri-o://ca79ca51f105055588eff90897589d93d24eb0e699ee4936ce7258e15cca61b6" gracePeriod=30
Nov 03 21:10:12.527102 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:12.527071    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:10:12.527486 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:12.527130    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48" podUID="d549e05f-a5a1-478c-bdb7-18b03af91b72" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:10:15.215669 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.212267    4993 patch_prober.go:29] interesting pod/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Nov 03 21:10:15.215669 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.212331    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r" podUID="122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Nov 03 21:10:15.372303 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.372274    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 21:10:15.397850 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.397822    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:10:15.418340 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.418316    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:10:15.452351 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452326    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-envoy\") pod \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") "
Nov 03 21:10:15.452492 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452357    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-workload-certs\") pod \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") "
Nov 03 21:10:15.452492 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452375    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-credential-socket\") pod \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") "
Nov 03 21:10:15.452492 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452394    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istiod-ca-cert\") pod \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") "
Nov 03 21:10:15.452492 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452411    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-podinfo\") pod \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") "
Nov 03 21:10:15.452492 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452445    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-kube-api-access-7djw9\" (UniqueName: \"kubernetes.io/projected/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-kube-api-access-7djw9\") pod \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") "
Nov 03 21:10:15.452492 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452475    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d549e05f-a5a1-478c-bdb7-18b03af91b72-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d549e05f-a5a1-478c-bdb7-18b03af91b72-istiod-ca-cert\") pod \"d549e05f-a5a1-478c-bdb7-18b03af91b72\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") "
Nov 03 21:10:15.452866 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452516    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-podinfo\") pod \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") "
Nov 03 21:10:15.452866 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452554    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-workload-certs\") pod \"d549e05f-a5a1-478c-bdb7-18b03af91b72\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") "
Nov 03 21:10:15.452866 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452589    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-token\" (UniqueName: \"kubernetes.io/projected/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-token\") pod \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") "
Nov 03 21:10:15.452866 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452648    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-workload-certs" pod "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" (UID: "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.452866 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452675    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-envoy\") pod \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") "
Nov 03 21:10:15.452866 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452734    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-token\" (UniqueName: \"kubernetes.io/projected/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-token\") pod \"d549e05f-a5a1-478c-bdb7-18b03af91b72\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") "
Nov 03 21:10:15.452866 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452744    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-credential-socket" pod "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" (UID: "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.452866 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452769    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-data\") pod \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") "
Nov 03 21:10:15.452866 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452807    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-podinfo\") pod \"d549e05f-a5a1-478c-bdb7-18b03af91b72\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") "
Nov 03 21:10:15.453357 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452810    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istiod-ca-cert" pod "298f7d43-f4d7-4108-b8a1-8df13e36ccc9" (UID: "298f7d43-f4d7-4108-b8a1-8df13e36ccc9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:10:15.453357 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.452838    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-workload-socket\") pod \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") "
Nov 03 21:10:15.453357 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.453011    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-token\" (UniqueName: \"kubernetes.io/projected/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-token\") pod \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") "
Nov 03 21:10:15.453357 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.453041    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-workload-socket" pod "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" (UID: "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.453357 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.453143    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d549e05f-a5a1-478c-bdb7-18b03af91b72-kube-api-access-l44s5\" (UniqueName: \"kubernetes.io/projected/d549e05f-a5a1-478c-bdb7-18b03af91b72-kube-api-access-l44s5\") pod \"d549e05f-a5a1-478c-bdb7-18b03af91b72\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") "
Nov 03 21:10:15.453357 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.453245    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d549e05f-a5a1-478c-bdb7-18b03af91b72-istiod-ca-cert" pod "d549e05f-a5a1-478c-bdb7-18b03af91b72" (UID: "d549e05f-a5a1-478c-bdb7-18b03af91b72"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:10:15.453357 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.453261    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-envoy\") pod \"d549e05f-a5a1-478c-bdb7-18b03af91b72\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") "
Nov 03 21:10:15.453357 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.453295    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-data\") pod \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") "
Nov 03 21:10:15.453357 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.453330    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-credential-socket\") pod \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") "
Nov 03 21:10:15.453876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.453370    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-data\") pod \"d549e05f-a5a1-478c-bdb7-18b03af91b72\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") "
Nov 03 21:10:15.453876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.453400    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-credential-socket\") pod \"d549e05f-a5a1-478c-bdb7-18b03af91b72\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") "
Nov 03 21:10:15.453876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.453430    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istiod-ca-cert\") pod \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") "
Nov 03 21:10:15.453876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.453481    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-kube-api-access-dkwmm\" (UniqueName: \"kubernetes.io/projected/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-kube-api-access-dkwmm\") pod \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\" (UID: \"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7\") "
Nov 03 21:10:15.453876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.453509    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-workload-certs\") pod \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") "
Nov 03 21:10:15.453876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.453665    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-workload-socket\") pod \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\" (UID: \"298f7d43-f4d7-4108-b8a1-8df13e36ccc9\") "
Nov 03 21:10:15.453876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.453704    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-workload-socket\") pod \"d549e05f-a5a1-478c-bdb7-18b03af91b72\" (UID: \"d549e05f-a5a1-478c-bdb7-18b03af91b72\") "
Nov 03 21:10:15.454424 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.454350    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.454424 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.454374    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.454424 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.454391    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.454424 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.454406    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d549e05f-a5a1-478c-bdb7-18b03af91b72-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.454424 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.454421    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.454898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.454683    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-workload-socket" pod "d549e05f-a5a1-478c-bdb7-18b03af91b72" (UID: "d549e05f-a5a1-478c-bdb7-18b03af91b72"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.455062 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.455036    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-envoy" pod "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" (UID: "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.455279 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.455253    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-kube-api-access-7djw9" pod "298f7d43-f4d7-4108-b8a1-8df13e36ccc9" (UID: "298f7d43-f4d7-4108-b8a1-8df13e36ccc9"). InnerVolumeSpecName "kube-api-access-7djw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:10:15.455495 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.455466    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-data" pod "d549e05f-a5a1-478c-bdb7-18b03af91b72" (UID: "d549e05f-a5a1-478c-bdb7-18b03af91b72"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.455617 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.455529    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-envoy" pod "298f7d43-f4d7-4108-b8a1-8df13e36ccc9" (UID: "298f7d43-f4d7-4108-b8a1-8df13e36ccc9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.455715 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.455675    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-token" pod "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" (UID: "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:10:15.455923 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.455869    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-workload-certs" pod "d549e05f-a5a1-478c-bdb7-18b03af91b72" (UID: "d549e05f-a5a1-478c-bdb7-18b03af91b72"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.456055 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.456035    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-workload-certs" pod "298f7d43-f4d7-4108-b8a1-8df13e36ccc9" (UID: "298f7d43-f4d7-4108-b8a1-8df13e36ccc9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.456254 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.456236    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-data" pod "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" (UID: "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.456763 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.456732    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-podinfo" pod "298f7d43-f4d7-4108-b8a1-8df13e36ccc9" (UID: "298f7d43-f4d7-4108-b8a1-8df13e36ccc9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:10:15.457258 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.456986    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-data" pod "298f7d43-f4d7-4108-b8a1-8df13e36ccc9" (UID: "298f7d43-f4d7-4108-b8a1-8df13e36ccc9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.457349 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.457300    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-token" pod "298f7d43-f4d7-4108-b8a1-8df13e36ccc9" (UID: "298f7d43-f4d7-4108-b8a1-8df13e36ccc9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:10:15.457520 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.457498    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-workload-socket" pod "298f7d43-f4d7-4108-b8a1-8df13e36ccc9" (UID: "298f7d43-f4d7-4108-b8a1-8df13e36ccc9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.457710 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.457692    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-credential-socket" pod "d549e05f-a5a1-478c-bdb7-18b03af91b72" (UID: "d549e05f-a5a1-478c-bdb7-18b03af91b72"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.458247 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.458010    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istiod-ca-cert" pod "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" (UID: "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:10:15.458247 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.458206    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-credential-socket" pod "298f7d43-f4d7-4108-b8a1-8df13e36ccc9" (UID: "298f7d43-f4d7-4108-b8a1-8df13e36ccc9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.458247 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.458212    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-kube-api-access-dkwmm" pod "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" (UID: "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7"). InnerVolumeSpecName "kube-api-access-dkwmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:10:15.458417 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.458243    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d549e05f-a5a1-478c-bdb7-18b03af91b72-kube-api-access-l44s5" pod "d549e05f-a5a1-478c-bdb7-18b03af91b72" (UID: "d549e05f-a5a1-478c-bdb7-18b03af91b72"). InnerVolumeSpecName "kube-api-access-l44s5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:10:15.458417 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.458346    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-podinfo" pod "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" (UID: "122dd6bc-5f41-4ad1-bd2c-568c4a79baa7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:10:15.458602 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.458582    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-token" pod "d549e05f-a5a1-478c-bdb7-18b03af91b72" (UID: "d549e05f-a5a1-478c-bdb7-18b03af91b72"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:10:15.458698 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.458648    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-envoy" pod "d549e05f-a5a1-478c-bdb7-18b03af91b72" (UID: "d549e05f-a5a1-478c-bdb7-18b03af91b72"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:10:15.459033 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.459014    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-podinfo" pod "d549e05f-a5a1-478c-bdb7-18b03af91b72" (UID: "d549e05f-a5a1-478c-bdb7-18b03af91b72"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:10:15.555696 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555500    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.555696 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555548    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.555696 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555564    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.555696 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555577    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.555696 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555591    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.555696 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555603    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.555696 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555617    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.555696 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555647    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.555696 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555663    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l44s5\" (UniqueName: \"kubernetes.io/projected/d549e05f-a5a1-478c-bdb7-18b03af91b72-kube-api-access-l44s5\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.555696 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555676    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.555696 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555691    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.555696 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555704    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.556432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555718    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.556432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555731    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.556432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555745    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.556432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555758    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkwmm\" (UniqueName: \"kubernetes.io/projected/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-kube-api-access-dkwmm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.556432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555771    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.556432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555783    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.556432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555798    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d549e05f-a5a1-478c-bdb7-18b03af91b72-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.556432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555812    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.556432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555826    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.556432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.555839    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7djw9\" (UniqueName: \"kubernetes.io/projected/298f7d43-f4d7-4108-b8a1-8df13e36ccc9-kube-api-access-7djw9\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:15.901586 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.901492    4993 generic.go:356] "Generic (PLEG): container finished" podID="298f7d43-f4d7-4108-b8a1-8df13e36ccc9" containerID="f3f07d04f2f506a38a084245252bb375517ec0dc3ab33de427e1622338a1fb66" exitCode=0
Nov 03 21:10:15.901586 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.901558    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"
Nov 03 21:10:15.901586 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.901574    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t" event={"ID":"298f7d43-f4d7-4108-b8a1-8df13e36ccc9","Type":"ContainerDied","Data":"f3f07d04f2f506a38a084245252bb375517ec0dc3ab33de427e1622338a1fb66"}
Nov 03 21:10:15.901916 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.901622    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t" event={"ID":"298f7d43-f4d7-4108-b8a1-8df13e36ccc9","Type":"ContainerDied","Data":"a32ef313a3a19ed2ff6b3b074e9605fa1e50a1a77bc2809205dab3554a6a9b51"}
Nov 03 21:10:15.901916 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.901665    4993 scope.go:122] "RemoveContainer" containerID="f3f07d04f2f506a38a084245252bb375517ec0dc3ab33de427e1622338a1fb66"
Nov 03 21:10:15.903505 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.903475    4993 generic.go:356] "Generic (PLEG): container finished" podID="122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" containerID="ca79ca51f105055588eff90897589d93d24eb0e699ee4936ce7258e15cca61b6" exitCode=0
Nov 03 21:10:15.903725 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.903554    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"
Nov 03 21:10:15.903725 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.903622    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r" event={"ID":"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7","Type":"ContainerDied","Data":"ca79ca51f105055588eff90897589d93d24eb0e699ee4936ce7258e15cca61b6"}
Nov 03 21:10:15.903960 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.903741    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r" event={"ID":"122dd6bc-5f41-4ad1-bd2c-568c4a79baa7","Type":"ContainerDied","Data":"ca78865d9b9c51fc2a0423962d7446549de0d3c2234fe6c60c187935de19cc65"}
Nov 03 21:10:15.905442 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.905422    4993 generic.go:356] "Generic (PLEG): container finished" podID="d549e05f-a5a1-478c-bdb7-18b03af91b72" containerID="61f101a971bbe6978267554a0f431161bb6d8c00c6d16cb9e0583a08f1cb40d6" exitCode=0
Nov 03 21:10:15.905515 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.905464    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48" event={"ID":"d549e05f-a5a1-478c-bdb7-18b03af91b72","Type":"ContainerDied","Data":"61f101a971bbe6978267554a0f431161bb6d8c00c6d16cb9e0583a08f1cb40d6"}
Nov 03 21:10:15.905515 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.905495    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"
Nov 03 21:10:15.905595 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.905496    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48" event={"ID":"d549e05f-a5a1-478c-bdb7-18b03af91b72","Type":"ContainerDied","Data":"57eb621be490be8aaa546961d39bccb19921b34898c06f5e64a080aea4ea04f5"}
Nov 03 21:10:15.913998 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.913983    4993 scope.go:122] "RemoveContainer" containerID="f3f07d04f2f506a38a084245252bb375517ec0dc3ab33de427e1622338a1fb66"
Nov 03 21:10:15.914316 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:10:15.914287    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3f07d04f2f506a38a084245252bb375517ec0dc3ab33de427e1622338a1fb66\": container with ID starting with f3f07d04f2f506a38a084245252bb375517ec0dc3ab33de427e1622338a1fb66 not found: ID does not exist" containerID="f3f07d04f2f506a38a084245252bb375517ec0dc3ab33de427e1622338a1fb66"
Nov 03 21:10:15.914370 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.914317    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3f07d04f2f506a38a084245252bb375517ec0dc3ab33de427e1622338a1fb66"} err="failed to get container status \"f3f07d04f2f506a38a084245252bb375517ec0dc3ab33de427e1622338a1fb66\": rpc error: code = NotFound desc = could not find container \"f3f07d04f2f506a38a084245252bb375517ec0dc3ab33de427e1622338a1fb66\": container with ID starting with f3f07d04f2f506a38a084245252bb375517ec0dc3ab33de427e1622338a1fb66 not found: ID does not exist"
Nov 03 21:10:15.914370 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.914333    4993 scope.go:122] "RemoveContainer" containerID="ca79ca51f105055588eff90897589d93d24eb0e699ee4936ce7258e15cca61b6"
Nov 03 21:10:15.922993 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.922963    4993 scope.go:122] "RemoveContainer" containerID="ca79ca51f105055588eff90897589d93d24eb0e699ee4936ce7258e15cca61b6"
Nov 03 21:10:15.923259 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:10:15.923239    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca79ca51f105055588eff90897589d93d24eb0e699ee4936ce7258e15cca61b6\": container with ID starting with ca79ca51f105055588eff90897589d93d24eb0e699ee4936ce7258e15cca61b6 not found: ID does not exist" containerID="ca79ca51f105055588eff90897589d93d24eb0e699ee4936ce7258e15cca61b6"
Nov 03 21:10:15.923303 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.923266    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca79ca51f105055588eff90897589d93d24eb0e699ee4936ce7258e15cca61b6"} err="failed to get container status \"ca79ca51f105055588eff90897589d93d24eb0e699ee4936ce7258e15cca61b6\": rpc error: code = NotFound desc = could not find container \"ca79ca51f105055588eff90897589d93d24eb0e699ee4936ce7258e15cca61b6\": container with ID starting with ca79ca51f105055588eff90897589d93d24eb0e699ee4936ce7258e15cca61b6 not found: ID does not exist"
Nov 03 21:10:15.923303 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.923284    4993 scope.go:122] "RemoveContainer" containerID="61f101a971bbe6978267554a0f431161bb6d8c00c6d16cb9e0583a08f1cb40d6"
Nov 03 21:10:15.933391 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.933357    4993 scope.go:122] "RemoveContainer" containerID="61f101a971bbe6978267554a0f431161bb6d8c00c6d16cb9e0583a08f1cb40d6"
Nov 03 21:10:15.933674 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:10:15.933653    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61f101a971bbe6978267554a0f431161bb6d8c00c6d16cb9e0583a08f1cb40d6\": container with ID starting with 61f101a971bbe6978267554a0f431161bb6d8c00c6d16cb9e0583a08f1cb40d6 not found: ID does not exist" containerID="61f101a971bbe6978267554a0f431161bb6d8c00c6d16cb9e0583a08f1cb40d6"
Nov 03 21:10:15.933757 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:15.933682    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61f101a971bbe6978267554a0f431161bb6d8c00c6d16cb9e0583a08f1cb40d6"} err="failed to get container status \"61f101a971bbe6978267554a0f431161bb6d8c00c6d16cb9e0583a08f1cb40d6\": rpc error: code = NotFound desc = could not find container \"61f101a971bbe6978267554a0f431161bb6d8c00c6d16cb9e0583a08f1cb40d6\": container with ID starting with 61f101a971bbe6978267554a0f431161bb6d8c00c6d16cb9e0583a08f1cb40d6 not found: ID does not exist"
Nov 03 21:10:16.067284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:16.067230    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"]
Nov 03 21:10:16.074889 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:16.074860    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-tqz48"]
Nov 03 21:10:16.181953 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:16.181874    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"]
Nov 03 21:10:16.187899 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:16.187869    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-f779t"]
Nov 03 21:10:16.294179 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:16.294145    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"]
Nov 03 21:10:16.301906 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:16.301882    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-89bb58-openshift-default-7f67949dc5-n762r"]
Nov 03 21:10:17.578829 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:17.578791    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" path="/var/lib/kubelet/pods/122dd6bc-5f41-4ad1-bd2c-568c4a79baa7/volumes"
Nov 03 21:10:17.579454 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:17.579430    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="298f7d43-f4d7-4108-b8a1-8df13e36ccc9" path="/var/lib/kubelet/pods/298f7d43-f4d7-4108-b8a1-8df13e36ccc9/volumes"
Nov 03 21:10:17.579982 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:17.579960    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d549e05f-a5a1-478c-bdb7-18b03af91b72" path="/var/lib/kubelet/pods/d549e05f-a5a1-478c-bdb7-18b03af91b72/volumes"
Nov 03 21:10:31.670534 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:10:31.670496    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:10:45.611213 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:45.611179    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"]
Nov 03 21:10:45.611621 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:45.611372    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz" podUID="609e8c8e-04cc-4f27-b80a-0ab170d8ab57" containerName="maas-api" containerID="cri-o://b9f1467779be4321b6a90fc6a22b6d54be0ac9479b7d408f8b19d7029bfe5de5" gracePeriod=30
Nov 03 21:10:45.917380 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:45.917342    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 21:10:46.022285 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.022232    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-metrics-certs\" (UniqueName: \"kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-metrics-certs\") pod \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") "
Nov 03 21:10:46.022483 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.022342    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-kube-api-access-pw869\" (UniqueName: \"kubernetes.io/projected/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-kube-api-access-pw869\") pod \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") "
Nov 03 21:10:46.022483 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.022445    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-maas-api-tls\") pod \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") "
Nov 03 21:10:46.022601 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.022489    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-openshift-service-ca\") pod \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\" (UID: \"609e8c8e-04cc-4f27-b80a-0ab170d8ab57\") "
Nov 03 21:10:46.023054 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.023010    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-openshift-service-ca" pod "609e8c8e-04cc-4f27-b80a-0ab170d8ab57" (UID: "609e8c8e-04cc-4f27-b80a-0ab170d8ab57"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:10:46.025824 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.025783    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-metrics-certs" pod "609e8c8e-04cc-4f27-b80a-0ab170d8ab57" (UID: "609e8c8e-04cc-4f27-b80a-0ab170d8ab57"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:10:46.025953 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.025821    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-kube-api-access-pw869" pod "609e8c8e-04cc-4f27-b80a-0ab170d8ab57" (UID: "609e8c8e-04cc-4f27-b80a-0ab170d8ab57"). InnerVolumeSpecName "kube-api-access-pw869". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:10:46.025953 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.025876    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-maas-api-tls" pod "609e8c8e-04cc-4f27-b80a-0ab170d8ab57" (UID: "609e8c8e-04cc-4f27-b80a-0ab170d8ab57"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:10:46.118474 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.118435    4993 generic.go:356] "Generic (PLEG): container finished" podID="609e8c8e-04cc-4f27-b80a-0ab170d8ab57" containerID="b9f1467779be4321b6a90fc6a22b6d54be0ac9479b7d408f8b19d7029bfe5de5" exitCode=0
Nov 03 21:10:46.118686 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.118480    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz" event={"ID":"609e8c8e-04cc-4f27-b80a-0ab170d8ab57","Type":"ContainerDied","Data":"b9f1467779be4321b6a90fc6a22b6d54be0ac9479b7d408f8b19d7029bfe5de5"}
Nov 03 21:10:46.118686 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.118520    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz" event={"ID":"609e8c8e-04cc-4f27-b80a-0ab170d8ab57","Type":"ContainerDied","Data":"094f0aad73b1c18d3fd44ef584b139711a386acd2249d7898a6f2c2c2276669b"}
Nov 03 21:10:46.118686 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.118540    4993 scope.go:122] "RemoveContainer" containerID="b9f1467779be4321b6a90fc6a22b6d54be0ac9479b7d408f8b19d7029bfe5de5"
Nov 03 21:10:46.118861 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.118698    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"
Nov 03 21:10:46.124163 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.124096    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pw869\" (UniqueName: \"kubernetes.io/projected/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-kube-api-access-pw869\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:46.124163 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.124129    4993 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-maas-api-tls\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:46.124163 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.124145    4993 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-openshift-service-ca\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:46.124163 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.124161    4993 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/609e8c8e-04cc-4f27-b80a-0ab170d8ab57-metrics-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:10:46.135143 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.135117    4993 scope.go:122] "RemoveContainer" containerID="b9f1467779be4321b6a90fc6a22b6d54be0ac9479b7d408f8b19d7029bfe5de5"
Nov 03 21:10:46.135483 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:10:46.135458    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9f1467779be4321b6a90fc6a22b6d54be0ac9479b7d408f8b19d7029bfe5de5\": container with ID starting with b9f1467779be4321b6a90fc6a22b6d54be0ac9479b7d408f8b19d7029bfe5de5 not found: ID does not exist" containerID="b9f1467779be4321b6a90fc6a22b6d54be0ac9479b7d408f8b19d7029bfe5de5"
Nov 03 21:10:46.135663 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.135491    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9f1467779be4321b6a90fc6a22b6d54be0ac9479b7d408f8b19d7029bfe5de5"} err="failed to get container status \"b9f1467779be4321b6a90fc6a22b6d54be0ac9479b7d408f8b19d7029bfe5de5\": rpc error: code = NotFound desc = could not find container \"b9f1467779be4321b6a90fc6a22b6d54be0ac9479b7d408f8b19d7029bfe5de5\": container with ID starting with b9f1467779be4321b6a90fc6a22b6d54be0ac9479b7d408f8b19d7029bfe5de5 not found: ID does not exist"
Nov 03 21:10:46.303983 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.303905    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"]
Nov 03 21:10:46.308073 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:46.308041    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-ca4642-64cf79c4cc-slzxz"]
Nov 03 21:10:47.580401 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:47.580365    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="609e8c8e-04cc-4f27-b80a-0ab170d8ab57" path="/var/lib/kubelet/pods/609e8c8e-04cc-4f27-b80a-0ab170d8ab57/volumes"
Nov 03 21:10:56.790264 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:56.790226    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"]
Nov 03 21:10:56.791100 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:56.791063    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq" podUID="4387fbe5-ef9e-4b48-9251-8181f1f941f9" containerName="istio-proxy" containerID="cri-o://c8a9fcb96c9a9a0025185b95b04aa95c64141c96781bc8f24f867041551cba00" gracePeriod=30
Nov 03 21:10:57.973590 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:57.973560    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:10:57.974006 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:10:57.973647    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq" podUID="4387fbe5-ef9e-4b48-9251-8181f1f941f9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:11:01.677964 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:11:01.677921    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:11:02.230743 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.230707    4993 generic.go:356] "Generic (PLEG): container finished" podID="4387fbe5-ef9e-4b48-9251-8181f1f941f9" containerID="c8a9fcb96c9a9a0025185b95b04aa95c64141c96781bc8f24f867041551cba00" exitCode=0
Nov 03 21:11:02.230932 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.230790    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq" event={"ID":"4387fbe5-ef9e-4b48-9251-8181f1f941f9","Type":"ContainerDied","Data":"c8a9fcb96c9a9a0025185b95b04aa95c64141c96781bc8f24f867041551cba00"}
Nov 03 21:11:02.478472 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.478443    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 21:11:02.536824 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.536752    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-podinfo\") pod \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") "
Nov 03 21:11:02.536824 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.536791    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-data\") pod \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") "
Nov 03 21:11:02.537042 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.536828    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-envoy\") pod \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") "
Nov 03 21:11:02.537042 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.536968    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-credential-socket\") pod \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") "
Nov 03 21:11:02.537042 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.537015    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-workload-certs\") pod \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") "
Nov 03 21:11:02.537204 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.537054    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istiod-ca-cert\") pod \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") "
Nov 03 21:11:02.537204 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.537122    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-data" pod "4387fbe5-ef9e-4b48-9251-8181f1f941f9" (UID: "4387fbe5-ef9e-4b48-9251-8181f1f941f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:02.537204 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.537155    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4387fbe5-ef9e-4b48-9251-8181f1f941f9-kube-api-access-jr9nc\" (UniqueName: \"kubernetes.io/projected/4387fbe5-ef9e-4b48-9251-8181f1f941f9-kube-api-access-jr9nc\") pod \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") "
Nov 03 21:11:02.537353 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.537221    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-credential-socket" pod "4387fbe5-ef9e-4b48-9251-8181f1f941f9" (UID: "4387fbe5-ef9e-4b48-9251-8181f1f941f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:02.537353 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.537238    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-workload-certs" pod "4387fbe5-ef9e-4b48-9251-8181f1f941f9" (UID: "4387fbe5-ef9e-4b48-9251-8181f1f941f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:02.537353 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.537226    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-token\" (UniqueName: \"kubernetes.io/projected/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-token\") pod \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") "
Nov 03 21:11:02.537517 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.537360    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-workload-socket\") pod \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\" (UID: \"4387fbe5-ef9e-4b48-9251-8181f1f941f9\") "
Nov 03 21:11:02.537517 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.537470    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istiod-ca-cert" pod "4387fbe5-ef9e-4b48-9251-8181f1f941f9" (UID: "4387fbe5-ef9e-4b48-9251-8181f1f941f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:02.537957 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.537865    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.537957 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.537892    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.537957 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.537907    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.537957 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.537922    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.538251 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.538003    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-workload-socket" pod "4387fbe5-ef9e-4b48-9251-8181f1f941f9" (UID: "4387fbe5-ef9e-4b48-9251-8181f1f941f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:02.539465 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.539414    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-podinfo" pod "4387fbe5-ef9e-4b48-9251-8181f1f941f9" (UID: "4387fbe5-ef9e-4b48-9251-8181f1f941f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:11:02.539695 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.539675    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4387fbe5-ef9e-4b48-9251-8181f1f941f9-kube-api-access-jr9nc" pod "4387fbe5-ef9e-4b48-9251-8181f1f941f9" (UID: "4387fbe5-ef9e-4b48-9251-8181f1f941f9"). InnerVolumeSpecName "kube-api-access-jr9nc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:02.539782 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.539761    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-token" pod "4387fbe5-ef9e-4b48-9251-8181f1f941f9" (UID: "4387fbe5-ef9e-4b48-9251-8181f1f941f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:02.539840 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.539794    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-envoy" pod "4387fbe5-ef9e-4b48-9251-8181f1f941f9" (UID: "4387fbe5-ef9e-4b48-9251-8181f1f941f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:02.639133 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.639073    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.639133 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.639107    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.639133 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.639124    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr9nc\" (UniqueName: \"kubernetes.io/projected/4387fbe5-ef9e-4b48-9251-8181f1f941f9-kube-api-access-jr9nc\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.639133 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.639139    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4387fbe5-ef9e-4b48-9251-8181f1f941f9-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:02.639468 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:02.639152    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4387fbe5-ef9e-4b48-9251-8181f1f941f9-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:03.237510 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:03.237476    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"
Nov 03 21:11:03.237510 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:03.237494    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq" event={"ID":"4387fbe5-ef9e-4b48-9251-8181f1f941f9","Type":"ContainerDied","Data":"1a9e0c02f852807d7dfaa8c6d61cec5066a80c553ec6ab557b98000479211f48"}
Nov 03 21:11:03.237999 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:03.237556    4993 scope.go:122] "RemoveContainer" containerID="c8a9fcb96c9a9a0025185b95b04aa95c64141c96781bc8f24f867041551cba00"
Nov 03 21:11:03.441403 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:03.441372    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"]
Nov 03 21:11:03.443648 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:03.443613    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-7gknq"]
Nov 03 21:11:03.578310 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:03.578227    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4387fbe5-ef9e-4b48-9251-8181f1f941f9" path="/var/lib/kubelet/pods/4387fbe5-ef9e-4b48-9251-8181f1f941f9/volumes"
Nov 03 21:11:18.311953 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:18.311912    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"]
Nov 03 21:11:18.312468 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:18.312175    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5" podUID="3fee816e-2cfa-4ba1-a6b1-a36260877599" containerName="maas-api" containerID="cri-o://440eb38a1e4299fe1b3ab4ac256cb7d51999a855eb45a141527bcb8bca0df9a0" gracePeriod=30
Nov 03 21:11:18.648235 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:18.648203    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 21:11:18.783772 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:18.779185    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-maas-api-tls\") pod \"3fee816e-2cfa-4ba1-a6b1-a36260877599\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") "
Nov 03 21:11:18.783772 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:18.779307    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3fee816e-2cfa-4ba1-a6b1-a36260877599-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fee816e-2cfa-4ba1-a6b1-a36260877599-openshift-service-ca\") pod \"3fee816e-2cfa-4ba1-a6b1-a36260877599\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") "
Nov 03 21:11:18.783772 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:18.779345    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3fee816e-2cfa-4ba1-a6b1-a36260877599-kube-api-access-dd7d5\" (UniqueName: \"kubernetes.io/projected/3fee816e-2cfa-4ba1-a6b1-a36260877599-kube-api-access-dd7d5\") pod \"3fee816e-2cfa-4ba1-a6b1-a36260877599\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") "
Nov 03 21:11:18.783772 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:18.779375    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-metrics-certs\") pod \"3fee816e-2cfa-4ba1-a6b1-a36260877599\" (UID: \"3fee816e-2cfa-4ba1-a6b1-a36260877599\") "
Nov 03 21:11:18.783772 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:18.781335    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fee816e-2cfa-4ba1-a6b1-a36260877599-openshift-service-ca" pod "3fee816e-2cfa-4ba1-a6b1-a36260877599" (UID: "3fee816e-2cfa-4ba1-a6b1-a36260877599"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:18.824836 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:18.824765    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-maas-api-tls" pod "3fee816e-2cfa-4ba1-a6b1-a36260877599" (UID: "3fee816e-2cfa-4ba1-a6b1-a36260877599"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:11:18.827855 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:18.827772    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fee816e-2cfa-4ba1-a6b1-a36260877599-kube-api-access-dd7d5" pod "3fee816e-2cfa-4ba1-a6b1-a36260877599" (UID: "3fee816e-2cfa-4ba1-a6b1-a36260877599"). InnerVolumeSpecName "kube-api-access-dd7d5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:18.831898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:18.831836    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-metrics-certs" pod "3fee816e-2cfa-4ba1-a6b1-a36260877599" (UID: "3fee816e-2cfa-4ba1-a6b1-a36260877599"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:11:18.881664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:18.881564    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd7d5\" (UniqueName: \"kubernetes.io/projected/3fee816e-2cfa-4ba1-a6b1-a36260877599-kube-api-access-dd7d5\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:18.881664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:18.881606    4993 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-metrics-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:18.881664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:18.881620    4993 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3fee816e-2cfa-4ba1-a6b1-a36260877599-maas-api-tls\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:18.881938 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:18.881670    4993 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3fee816e-2cfa-4ba1-a6b1-a36260877599-openshift-service-ca\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:19.344599 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:19.344561    4993 generic.go:356] "Generic (PLEG): container finished" podID="3fee816e-2cfa-4ba1-a6b1-a36260877599" containerID="440eb38a1e4299fe1b3ab4ac256cb7d51999a855eb45a141527bcb8bca0df9a0" exitCode=0
Nov 03 21:11:19.345133 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:19.344616    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5" event={"ID":"3fee816e-2cfa-4ba1-a6b1-a36260877599","Type":"ContainerDied","Data":"440eb38a1e4299fe1b3ab4ac256cb7d51999a855eb45a141527bcb8bca0df9a0"}
Nov 03 21:11:19.345133 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:19.344685    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5" event={"ID":"3fee816e-2cfa-4ba1-a6b1-a36260877599","Type":"ContainerDied","Data":"cfd809bfaf7d2f854d9c4cf18dac34a5b052d0a30609cda05405a11f0643a814"}
Nov 03 21:11:19.345133 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:19.344717    4993 scope.go:122] "RemoveContainer" containerID="440eb38a1e4299fe1b3ab4ac256cb7d51999a855eb45a141527bcb8bca0df9a0"
Nov 03 21:11:19.345133 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:19.344910    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"
Nov 03 21:11:19.366417 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:19.365818    4993 scope.go:122] "RemoveContainer" containerID="440eb38a1e4299fe1b3ab4ac256cb7d51999a855eb45a141527bcb8bca0df9a0"
Nov 03 21:11:19.366543 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:11:19.366474    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"440eb38a1e4299fe1b3ab4ac256cb7d51999a855eb45a141527bcb8bca0df9a0\": container with ID starting with 440eb38a1e4299fe1b3ab4ac256cb7d51999a855eb45a141527bcb8bca0df9a0 not found: ID does not exist" containerID="440eb38a1e4299fe1b3ab4ac256cb7d51999a855eb45a141527bcb8bca0df9a0"
Nov 03 21:11:19.366543 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:19.366512    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"440eb38a1e4299fe1b3ab4ac256cb7d51999a855eb45a141527bcb8bca0df9a0"} err="failed to get container status \"440eb38a1e4299fe1b3ab4ac256cb7d51999a855eb45a141527bcb8bca0df9a0\": rpc error: code = NotFound desc = could not find container \"440eb38a1e4299fe1b3ab4ac256cb7d51999a855eb45a141527bcb8bca0df9a0\": container with ID starting with 440eb38a1e4299fe1b3ab4ac256cb7d51999a855eb45a141527bcb8bca0df9a0 not found: ID does not exist"
Nov 03 21:11:19.514956 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:19.514920    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"]
Nov 03 21:11:19.563512 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:19.563479    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-499187-6454d55477-sf6x5"]
Nov 03 21:11:19.578096 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:19.578054    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3fee816e-2cfa-4ba1-a6b1-a36260877599" path="/var/lib/kubelet/pods/3fee816e-2cfa-4ba1-a6b1-a36260877599/volumes"
Nov 03 21:11:29.110179 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:29.110123    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"]
Nov 03 21:11:29.111140 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:29.111106    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" podUID="0ceb5aa1-a956-4442-b710-36453c7a1e79" containerName="istio-proxy" containerID="cri-o://dd5b31adfc0ad4feb45a7383ace9f9f20737d7b81f710c0df4436f01821f539a" gracePeriod=30
Nov 03 21:11:29.126916 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:29.126883    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"]
Nov 03 21:11:29.127193 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:29.127145    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" podUID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" containerName="istio-proxy" containerID="cri-o://5fd160bdd9e28a134219553c915944b93f4f567aeffcf1aaee966416153cd931" gracePeriod=30
Nov 03 21:11:29.150986 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:29.150955    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"]
Nov 03 21:11:29.151189 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:29.151154    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552" podUID="be1af043-dda6-4590-8f69-36c2abfcd868" containerName="istio-proxy" containerID="cri-o://2bccd6f3c7127d7882de17ee864298e584a576e4c88c2b953eeefe38609e8121" gracePeriod=30
Nov 03 21:11:31.671434 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:11:31.671398    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:11:32.042337 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:32.042296    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:11:32.042525 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:32.042376    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" podUID="0ceb5aa1-a956-4442-b710-36453c7a1e79" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:11:32.198533 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:32.198489    4993 patch_prober.go:29] interesting pod/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:11:32.198707 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:32.198570    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" podUID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:11:34.473411 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.473163    4993 generic.go:356] "Generic (PLEG): container finished" podID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" containerID="5fd160bdd9e28a134219553c915944b93f4f567aeffcf1aaee966416153cd931" exitCode=0
Nov 03 21:11:34.473411 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.473295    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" event={"ID":"996b6dcf-8250-4c0e-8144-b8cbb85a2b25","Type":"ContainerDied","Data":"5fd160bdd9e28a134219553c915944b93f4f567aeffcf1aaee966416153cd931"}
Nov 03 21:11:34.477515 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.477288    4993 generic.go:356] "Generic (PLEG): container finished" podID="0ceb5aa1-a956-4442-b710-36453c7a1e79" containerID="dd5b31adfc0ad4feb45a7383ace9f9f20737d7b81f710c0df4436f01821f539a" exitCode=0
Nov 03 21:11:34.477515 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.477427    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" event={"ID":"0ceb5aa1-a956-4442-b710-36453c7a1e79","Type":"ContainerDied","Data":"dd5b31adfc0ad4feb45a7383ace9f9f20737d7b81f710c0df4436f01821f539a"}
Nov 03 21:11:34.480277 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.480245    4993 generic.go:356] "Generic (PLEG): container finished" podID="be1af043-dda6-4590-8f69-36c2abfcd868" containerID="2bccd6f3c7127d7882de17ee864298e584a576e4c88c2b953eeefe38609e8121" exitCode=0
Nov 03 21:11:34.480417 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.480352    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552" event={"ID":"be1af043-dda6-4590-8f69-36c2abfcd868","Type":"ContainerDied","Data":"2bccd6f3c7127d7882de17ee864298e584a576e4c88c2b953eeefe38609e8121"}
Nov 03 21:11:34.480417 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.480404    4993 scope.go:122] "RemoveContainer" containerID="bed8089f07c51ef3556204d691ccb9e523641ed071daa260b8c810b30f06cf89"
Nov 03 21:11:34.555947 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.555916    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:11:34.579958 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.579923    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:11:34.591987 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.591960    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 21:11:34.641312 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.641274    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-workload-certs\") pod \"0ceb5aa1-a956-4442-b710-36453c7a1e79\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") "
Nov 03 21:11:34.641478 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.641421    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-kube-api-access-vdn6c\" (UniqueName: \"kubernetes.io/projected/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-kube-api-access-vdn6c\") pod \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") "
Nov 03 21:11:34.641478 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.641451    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-envoy\") pod \"0ceb5aa1-a956-4442-b710-36453c7a1e79\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") "
Nov 03 21:11:34.641610 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.641494    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-credential-socket\") pod \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") "
Nov 03 21:11:34.641610 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.641540    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-workload-certs" pod "0ceb5aa1-a956-4442-b710-36453c7a1e79" (UID: "0ceb5aa1-a956-4442-b710-36453c7a1e79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.641610 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.641596    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-podinfo\") pod \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") "
Nov 03 21:11:34.641819 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.641697    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/be1af043-dda6-4590-8f69-36c2abfcd868-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be1af043-dda6-4590-8f69-36c2abfcd868-istio-podinfo\") pod \"be1af043-dda6-4590-8f69-36c2abfcd868\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") "
Nov 03 21:11:34.641819 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.641771    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be1af043-dda6-4590-8f69-36c2abfcd868-kube-api-access-9xvbd\" (UniqueName: \"kubernetes.io/projected/be1af043-dda6-4590-8f69-36c2abfcd868-kube-api-access-9xvbd\") pod \"be1af043-dda6-4590-8f69-36c2abfcd868\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") "
Nov 03 21:11:34.641819 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.641806    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-credential-socket" pod "996b6dcf-8250-4c0e-8144-b8cbb85a2b25" (UID: "996b6dcf-8250-4c0e-8144-b8cbb85a2b25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.641973 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.641856    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-data\") pod \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") "
Nov 03 21:11:34.641973 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.641916    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be1af043-dda6-4590-8f69-36c2abfcd868-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be1af043-dda6-4590-8f69-36c2abfcd868-istiod-ca-cert\") pod \"be1af043-dda6-4590-8f69-36c2abfcd868\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") "
Nov 03 21:11:34.641973 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.641964    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be1af043-dda6-4590-8f69-36c2abfcd868-istio-token\" (UniqueName: \"kubernetes.io/projected/be1af043-dda6-4590-8f69-36c2abfcd868-istio-token\") pod \"be1af043-dda6-4590-8f69-36c2abfcd868\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") "
Nov 03 21:11:34.642132 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642098    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-podinfo\") pod \"0ceb5aa1-a956-4442-b710-36453c7a1e79\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") "
Nov 03 21:11:34.642194 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642132    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-credential-socket\") pod \"0ceb5aa1-a956-4442-b710-36453c7a1e79\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") "
Nov 03 21:11:34.642194 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642150    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-data" pod "996b6dcf-8250-4c0e-8144-b8cbb85a2b25" (UID: "996b6dcf-8250-4c0e-8144-b8cbb85a2b25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.642294 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642196    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-istio-data\") pod \"be1af043-dda6-4590-8f69-36c2abfcd868\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") "
Nov 03 21:11:34.642354 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642292    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be1af043-dda6-4590-8f69-36c2abfcd868-istiod-ca-cert" pod "be1af043-dda6-4590-8f69-36c2abfcd868" (UID: "be1af043-dda6-4590-8f69-36c2abfcd868"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:34.642410 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642359    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ceb5aa1-a956-4442-b710-36453c7a1e79-kube-api-access-qkj2b\" (UniqueName: \"kubernetes.io/projected/0ceb5aa1-a956-4442-b710-36453c7a1e79-kube-api-access-qkj2b\") pod \"0ceb5aa1-a956-4442-b710-36453c7a1e79\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") "
Nov 03 21:11:34.642410 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642400    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-token\" (UniqueName: \"kubernetes.io/projected/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-token\") pod \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") "
Nov 03 21:11:34.642695 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642450    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-istio-envoy\") pod \"be1af043-dda6-4590-8f69-36c2abfcd868\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") "
Nov 03 21:11:34.642695 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642485    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ceb5aa1-a956-4442-b710-36453c7a1e79-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ceb5aa1-a956-4442-b710-36453c7a1e79-istiod-ca-cert\") pod \"0ceb5aa1-a956-4442-b710-36453c7a1e79\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") "
Nov 03 21:11:34.642695 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642514    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-workload-certs\") pod \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") "
Nov 03 21:11:34.642695 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642545    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-workload-socket\") pod \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") "
Nov 03 21:11:34.642695 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642581    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-credential-socket" pod "0ceb5aa1-a956-4442-b710-36453c7a1e79" (UID: "0ceb5aa1-a956-4442-b710-36453c7a1e79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.642695 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642587    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-credential-socket\") pod \"be1af043-dda6-4590-8f69-36c2abfcd868\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") "
Nov 03 21:11:34.642695 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642683    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-token\" (UniqueName: \"kubernetes.io/projected/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-token\") pod \"0ceb5aa1-a956-4442-b710-36453c7a1e79\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") "
Nov 03 21:11:34.643134 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642717    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-workload-socket\") pod \"be1af043-dda6-4590-8f69-36c2abfcd868\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") "
Nov 03 21:11:34.643134 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642851    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-credential-socket" pod "be1af043-dda6-4590-8f69-36c2abfcd868" (UID: "be1af043-dda6-4590-8f69-36c2abfcd868"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.643134 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.642958    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-workload-socket\") pod \"0ceb5aa1-a956-4442-b710-36453c7a1e79\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") "
Nov 03 21:11:34.643134 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643005    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-envoy\") pod \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") "
Nov 03 21:11:34.643134 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643043    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-istio-data" pod "be1af043-dda6-4590-8f69-36c2abfcd868" (UID: "be1af043-dda6-4590-8f69-36c2abfcd868"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.643134 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643059    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-workload-certs\") pod \"be1af043-dda6-4590-8f69-36c2abfcd868\" (UID: \"be1af043-dda6-4590-8f69-36c2abfcd868\") "
Nov 03 21:11:34.643134 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643109    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-data\") pod \"0ceb5aa1-a956-4442-b710-36453c7a1e79\" (UID: \"0ceb5aa1-a956-4442-b710-36453c7a1e79\") "
Nov 03 21:11:34.643493 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643158    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istiod-ca-cert\") pod \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\" (UID: \"996b6dcf-8250-4c0e-8144-b8cbb85a2b25\") "
Nov 03 21:11:34.643493 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643330    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ceb5aa1-a956-4442-b710-36453c7a1e79-istiod-ca-cert" pod "0ceb5aa1-a956-4442-b710-36453c7a1e79" (UID: "0ceb5aa1-a956-4442-b710-36453c7a1e79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:34.643493 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643372    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-workload-socket" pod "be1af043-dda6-4590-8f69-36c2abfcd868" (UID: "be1af043-dda6-4590-8f69-36c2abfcd868"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.643687 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643657    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-workload-certs" pod "996b6dcf-8250-4c0e-8144-b8cbb85a2b25" (UID: "996b6dcf-8250-4c0e-8144-b8cbb85a2b25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.643748 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643706    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istiod-ca-cert" pod "996b6dcf-8250-4c0e-8144-b8cbb85a2b25" (UID: "996b6dcf-8250-4c0e-8144-b8cbb85a2b25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:11:34.643877 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643851    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.643997 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643876    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-workload-socket" pod "996b6dcf-8250-4c0e-8144-b8cbb85a2b25" (UID: "996b6dcf-8250-4c0e-8144-b8cbb85a2b25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.643997 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643885    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.643997 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643900    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be1af043-dda6-4590-8f69-36c2abfcd868-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.643997 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643914    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.643997 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643925    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.643997 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643938    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ceb5aa1-a956-4442-b710-36453c7a1e79-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.643997 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643951    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.643997 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643964    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.643997 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643977    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.643997 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.643990    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.645361 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.644017    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-workload-certs" pod "be1af043-dda6-4590-8f69-36c2abfcd868" (UID: "be1af043-dda6-4590-8f69-36c2abfcd868"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.645361 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.644076    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-workload-socket" pod "0ceb5aa1-a956-4442-b710-36453c7a1e79" (UID: "0ceb5aa1-a956-4442-b710-36453c7a1e79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.645361 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.644537    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-data" pod "0ceb5aa1-a956-4442-b710-36453c7a1e79" (UID: "0ceb5aa1-a956-4442-b710-36453c7a1e79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.645361 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.644809    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/be1af043-dda6-4590-8f69-36c2abfcd868-istio-podinfo" pod "be1af043-dda6-4590-8f69-36c2abfcd868" (UID: "be1af043-dda6-4590-8f69-36c2abfcd868"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:11:34.645361 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.644965    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-podinfo" pod "996b6dcf-8250-4c0e-8144-b8cbb85a2b25" (UID: "996b6dcf-8250-4c0e-8144-b8cbb85a2b25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:11:34.646008 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.645987    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-podinfo" pod "0ceb5aa1-a956-4442-b710-36453c7a1e79" (UID: "0ceb5aa1-a956-4442-b710-36453c7a1e79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:11:34.647081 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.647032    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be1af043-dda6-4590-8f69-36c2abfcd868-istio-token" pod "be1af043-dda6-4590-8f69-36c2abfcd868" (UID: "be1af043-dda6-4590-8f69-36c2abfcd868"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:34.647081 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.647048    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-envoy" pod "0ceb5aa1-a956-4442-b710-36453c7a1e79" (UID: "0ceb5aa1-a956-4442-b710-36453c7a1e79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.647081 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.647049    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-token" pod "996b6dcf-8250-4c0e-8144-b8cbb85a2b25" (UID: "996b6dcf-8250-4c0e-8144-b8cbb85a2b25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:34.647081 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.647065    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ceb5aa1-a956-4442-b710-36453c7a1e79-kube-api-access-qkj2b" pod "0ceb5aa1-a956-4442-b710-36453c7a1e79" (UID: "0ceb5aa1-a956-4442-b710-36453c7a1e79"). InnerVolumeSpecName "kube-api-access-qkj2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:34.647302 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.647134    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-token" pod "0ceb5aa1-a956-4442-b710-36453c7a1e79" (UID: "0ceb5aa1-a956-4442-b710-36453c7a1e79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:34.647302 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.647225    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-envoy" pod "996b6dcf-8250-4c0e-8144-b8cbb85a2b25" (UID: "996b6dcf-8250-4c0e-8144-b8cbb85a2b25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.647302 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.647225    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-kube-api-access-vdn6c" pod "996b6dcf-8250-4c0e-8144-b8cbb85a2b25" (UID: "996b6dcf-8250-4c0e-8144-b8cbb85a2b25"). InnerVolumeSpecName "kube-api-access-vdn6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:34.647400 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.647339    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be1af043-dda6-4590-8f69-36c2abfcd868-kube-api-access-9xvbd" pod "be1af043-dda6-4590-8f69-36c2abfcd868" (UID: "be1af043-dda6-4590-8f69-36c2abfcd868"). InnerVolumeSpecName "kube-api-access-9xvbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:11:34.648169 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.648149    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-istio-envoy" pod "be1af043-dda6-4590-8f69-36c2abfcd868" (UID: "be1af043-dda6-4590-8f69-36c2abfcd868"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:11:34.745432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745392    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.745432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745434    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.745655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745449    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.745655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745463    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.745655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745477    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.745655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745492    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.745655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745505    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be1af043-dda6-4590-8f69-36c2abfcd868-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.745655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745517    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.745655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745531    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdn6c\" (UniqueName: \"kubernetes.io/projected/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-kube-api-access-vdn6c\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.745655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745561    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.745655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745574    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.745655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745589    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be1af043-dda6-4590-8f69-36c2abfcd868-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.745655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745602    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xvbd\" (UniqueName: \"kubernetes.io/projected/be1af043-dda6-4590-8f69-36c2abfcd868-kube-api-access-9xvbd\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.745655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745614    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be1af043-dda6-4590-8f69-36c2abfcd868-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.745655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745645    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ceb5aa1-a956-4442-b710-36453c7a1e79-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.746065 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745661    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkj2b\" (UniqueName: \"kubernetes.io/projected/0ceb5aa1-a956-4442-b710-36453c7a1e79-kube-api-access-qkj2b\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:34.746065 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:34.745674    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/996b6dcf-8250-4c0e-8144-b8cbb85a2b25-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:11:35.487559 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:35.487522    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"
Nov 03 21:11:35.488021 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:35.487519    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q" event={"ID":"996b6dcf-8250-4c0e-8144-b8cbb85a2b25","Type":"ContainerDied","Data":"ce2309ea8eb1deeebd0b6e71bbe20f9fb6fddcc26b11c5589fcead01b4993314"}
Nov 03 21:11:35.488021 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:35.487695    4993 scope.go:122] "RemoveContainer" containerID="5fd160bdd9e28a134219553c915944b93f4f567aeffcf1aaee966416153cd931"
Nov 03 21:11:35.488945 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:35.488926    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"
Nov 03 21:11:35.489042 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:35.488973    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb" event={"ID":"0ceb5aa1-a956-4442-b710-36453c7a1e79","Type":"ContainerDied","Data":"5a5cdf4e927eace42dd816e80f6b3618ba49a31739e647a2e134f0c5d6b012a1"}
Nov 03 21:11:35.490759 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:35.490728    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552" event={"ID":"be1af043-dda6-4590-8f69-36c2abfcd868","Type":"ContainerDied","Data":"7c14f3bc6209587b2943409d7b0f8362e74eacaf59e52d3e8ab44506dd910715"}
Nov 03 21:11:35.490759 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:35.490742    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"
Nov 03 21:11:35.500871 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:35.500852    4993 scope.go:122] "RemoveContainer" containerID="dd5b31adfc0ad4feb45a7383ace9f9f20737d7b81f710c0df4436f01821f539a"
Nov 03 21:11:35.510675 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:35.510655    4993 scope.go:122] "RemoveContainer" containerID="2bccd6f3c7127d7882de17ee864298e584a576e4c88c2b953eeefe38609e8121"
Nov 03 21:11:35.701699 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:35.701660    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"]
Nov 03 21:11:35.750994 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:35.750922    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-h9552"]
Nov 03 21:11:35.814576 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:35.814544    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"]
Nov 03 21:11:35.823367 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:35.823337    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-thbtb"]
Nov 03 21:11:35.931853 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:35.931815    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"]
Nov 03 21:11:35.938751 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:35.938726    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-499187-openshift-default-c6598c9fc-gqr9q"]
Nov 03 21:11:37.577462 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:37.577431    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ceb5aa1-a956-4442-b710-36453c7a1e79" path="/var/lib/kubelet/pods/0ceb5aa1-a956-4442-b710-36453c7a1e79/volumes"
Nov 03 21:11:37.577922 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:37.577908    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" path="/var/lib/kubelet/pods/996b6dcf-8250-4c0e-8144-b8cbb85a2b25/volumes"
Nov 03 21:11:37.578320 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:37.578307    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be1af043-dda6-4590-8f69-36c2abfcd868" path="/var/lib/kubelet/pods/be1af043-dda6-4590-8f69-36c2abfcd868/volumes"
Nov 03 21:11:56.177087 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:56.177051    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"]
Nov 03 21:11:56.177658 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:56.177299    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw" podUID="eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" containerName="istio-proxy" containerID="cri-o://7f86e68a1e3a1d14f75ecdd4554ba453fc4fdb4a5ce86b2e4ef9928d5137c87c" gracePeriod=30
Nov 03 21:11:58.434211 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:58.434141    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:11:58.434211 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:11:58.434192    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw" podUID="eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:12:01.441544 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.441484    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 21:12:01.488320 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.488281    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-token\" (UniqueName: \"kubernetes.io/projected/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-token\") pod \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") "
Nov 03 21:12:01.488545 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.488335    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-workload-certs\") pod \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") "
Nov 03 21:12:01.488545 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.488361    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-credential-socket\") pod \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") "
Nov 03 21:12:01.488545 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.488466    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-workload-socket\") pod \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") "
Nov 03 21:12:01.488545 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.488539    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-podinfo\") pod \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") "
Nov 03 21:12:01.488793 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.488581    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-envoy\") pod \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") "
Nov 03 21:12:01.488793 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.488674    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-workload-certs" pod "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" (UID: "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:12:01.488793 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.488692    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-credential-socket" pod "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" (UID: "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:12:01.488793 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.488708    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-workload-socket" pod "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" (UID: "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:12:01.488793 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.488697    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-data\") pod \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") "
Nov 03 21:12:01.489054 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.488795    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istiod-ca-cert\") pod \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") "
Nov 03 21:12:01.489054 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.488884    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-data" pod "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" (UID: "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:12:01.489054 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.488909    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-kube-api-access-crjj5\" (UniqueName: \"kubernetes.io/projected/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-kube-api-access-crjj5\") pod \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\" (UID: \"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7\") "
Nov 03 21:12:01.489206 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.489190    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istiod-ca-cert" pod "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" (UID: "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:12:01.489592 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.489565    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:01.489770 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.489595    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:01.489770 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.489612    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:01.489770 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.489649    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:01.489770 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.489665    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:01.490847 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.490814    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-token" pod "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" (UID: "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:12:01.490969 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.490895    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-envoy" pod "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" (UID: "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:12:01.491583 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.491559    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-kube-api-access-crjj5" pod "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" (UID: "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7"). InnerVolumeSpecName "kube-api-access-crjj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:12:01.491583 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.491567    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-podinfo" pod "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" (UID: "eb50b6fe-df28-4b1d-8a77-f7cdecf833c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:12:01.591433 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.591402    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crjj5\" (UniqueName: \"kubernetes.io/projected/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-kube-api-access-crjj5\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:01.591560 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.591437    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:01.591560 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.591452    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:01.591560 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.591465    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:01.655554 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.655510    4993 generic.go:356] "Generic (PLEG): container finished" podID="eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" containerID="7f86e68a1e3a1d14f75ecdd4554ba453fc4fdb4a5ce86b2e4ef9928d5137c87c" exitCode=0
Nov 03 21:12:01.655780 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.655595    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"
Nov 03 21:12:01.655780 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.655593    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw" event={"ID":"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7","Type":"ContainerDied","Data":"7f86e68a1e3a1d14f75ecdd4554ba453fc4fdb4a5ce86b2e4ef9928d5137c87c"}
Nov 03 21:12:01.655780 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.655682    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw" event={"ID":"eb50b6fe-df28-4b1d-8a77-f7cdecf833c7","Type":"ContainerDied","Data":"45b1d181d8ac540e11d9ff76bb8ef08062b6f24ebcae675809f016ab017496c0"}
Nov 03 21:12:01.655780 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.655718    4993 scope.go:122] "RemoveContainer" containerID="7f86e68a1e3a1d14f75ecdd4554ba453fc4fdb4a5ce86b2e4ef9928d5137c87c"
Nov 03 21:12:01.666521 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.666438    4993 scope.go:122] "RemoveContainer" containerID="7f86e68a1e3a1d14f75ecdd4554ba453fc4fdb4a5ce86b2e4ef9928d5137c87c"
Nov 03 21:12:01.667038 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:01.667012    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f86e68a1e3a1d14f75ecdd4554ba453fc4fdb4a5ce86b2e4ef9928d5137c87c\": container with ID starting with 7f86e68a1e3a1d14f75ecdd4554ba453fc4fdb4a5ce86b2e4ef9928d5137c87c not found: ID does not exist" containerID="7f86e68a1e3a1d14f75ecdd4554ba453fc4fdb4a5ce86b2e4ef9928d5137c87c"
Nov 03 21:12:01.667124 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.667041    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f86e68a1e3a1d14f75ecdd4554ba453fc4fdb4a5ce86b2e4ef9928d5137c87c"} err="failed to get container status \"7f86e68a1e3a1d14f75ecdd4554ba453fc4fdb4a5ce86b2e4ef9928d5137c87c\": rpc error: code = NotFound desc = could not find container \"7f86e68a1e3a1d14f75ecdd4554ba453fc4fdb4a5ce86b2e4ef9928d5137c87c\": container with ID starting with 7f86e68a1e3a1d14f75ecdd4554ba453fc4fdb4a5ce86b2e4ef9928d5137c87c not found: ID does not exist"
Nov 03 21:12:01.675534 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:01.675500    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:12:01.819728 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.819694    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"]
Nov 03 21:12:01.826641 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:01.826587    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-mtlxw"]
Nov 03 21:12:03.578956 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:03.578910    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" path="/var/lib/kubelet/pods/eb50b6fe-df28-4b1d-8a77-f7cdecf833c7/volumes"
Nov 03 21:12:25.966048 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966012    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7364-6e0ed0"]
Nov 03 21:12:25.966755 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966733    4993 cpu_manager.go:414] "containerMap: removing container" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main"
Nov 03 21:12:25.966824 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966759    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main"
Nov 03 21:12:25.966824 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966776    4993 cpu_manager.go:414] "containerMap: removing container" podUID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" containerName="istio-proxy"
Nov 03 21:12:25.966824 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966784    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" containerName="istio-proxy"
Nov 03 21:12:25.966824 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966797    4993 cpu_manager.go:414] "containerMap: removing container" podUID="260b8942-a2ee-41fa-b64d-daa30ecc0bc1" containerName="istio-proxy"
Nov 03 21:12:25.966824 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966806    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="260b8942-a2ee-41fa-b64d-daa30ecc0bc1" containerName="istio-proxy"
Nov 03 21:12:25.966824 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966821    4993 cpu_manager.go:414] "containerMap: removing container" podUID="be1af043-dda6-4590-8f69-36c2abfcd868" containerName="istio-proxy"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966828    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be1af043-dda6-4590-8f69-36c2abfcd868" containerName="istio-proxy"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966840    4993 cpu_manager.go:414] "containerMap: removing container" podUID="be1af043-dda6-4590-8f69-36c2abfcd868" containerName="istio-proxy"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966847    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be1af043-dda6-4590-8f69-36c2abfcd868" containerName="istio-proxy"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966855    4993 cpu_manager.go:414] "containerMap: removing container" podUID="d549e05f-a5a1-478c-bdb7-18b03af91b72" containerName="istio-proxy"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966862    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d549e05f-a5a1-478c-bdb7-18b03af91b72" containerName="istio-proxy"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966875    4993 cpu_manager.go:414] "containerMap: removing container" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966884    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966900    4993 cpu_manager.go:414] "containerMap: removing container" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966907    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966917    4993 cpu_manager.go:414] "containerMap: removing container" podUID="122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" containerName="istio-proxy"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966924    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" containerName="istio-proxy"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966939    4993 cpu_manager.go:414] "containerMap: removing container" podUID="4387fbe5-ef9e-4b48-9251-8181f1f941f9" containerName="istio-proxy"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966945    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4387fbe5-ef9e-4b48-9251-8181f1f941f9" containerName="istio-proxy"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966959    4993 cpu_manager.go:414] "containerMap: removing container" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966967    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966983    4993 cpu_manager.go:414] "containerMap: removing container" podUID="3fee816e-2cfa-4ba1-a6b1-a36260877599" containerName="maas-api"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.966990    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3fee816e-2cfa-4ba1-a6b1-a36260877599" containerName="maas-api"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967002    4993 cpu_manager.go:414] "containerMap: removing container" podUID="76814bbc-dc45-440a-947c-1f4979ad86de" containerName="istio-proxy"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967008    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76814bbc-dc45-440a-947c-1f4979ad86de" containerName="istio-proxy"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967025    4993 cpu_manager.go:414] "containerMap: removing container" podUID="298f7d43-f4d7-4108-b8a1-8df13e36ccc9" containerName="istio-proxy"
Nov 03 21:12:25.967027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967032    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="298f7d43-f4d7-4108-b8a1-8df13e36ccc9" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967042    4993 cpu_manager.go:414] "containerMap: removing container" podUID="82ef457f-b231-432d-9c89-52e22180cea2" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967050    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82ef457f-b231-432d-9c89-52e22180cea2" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967062    4993 cpu_manager.go:414] "containerMap: removing container" podUID="eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967068    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967078    4993 cpu_manager.go:414] "containerMap: removing container" podUID="0ceb5aa1-a956-4442-b710-36453c7a1e79" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967086    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ceb5aa1-a956-4442-b710-36453c7a1e79" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967096    4993 cpu_manager.go:414] "containerMap: removing container" podUID="5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967103    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967126    4993 cpu_manager.go:414] "containerMap: removing container" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967133    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967152    4993 cpu_manager.go:414] "containerMap: removing container" podUID="b380cf76-c519-4719-84fa-5c3025ea84f1" containerName="maas-api"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967158    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b380cf76-c519-4719-84fa-5c3025ea84f1" containerName="maas-api"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967180    4993 cpu_manager.go:414] "containerMap: removing container" podUID="609e8c8e-04cc-4f27-b80a-0ab170d8ab57" containerName="maas-api"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967187    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="609e8c8e-04cc-4f27-b80a-0ab170d8ab57" containerName="maas-api"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967196    4993 cpu_manager.go:414] "containerMap: removing container" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967203    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967325    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf6e552c-52c9-49be-8dc5-4c2d3dde5a2e" containerName="main"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967337    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="996b6dcf-8250-4c0e-8144-b8cbb85a2b25" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967347    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="260b8942-a2ee-41fa-b64d-daa30ecc0bc1" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967357    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="be1af043-dda6-4590-8f69-36c2abfcd868" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967366    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="be1af043-dda6-4590-8f69-36c2abfcd868" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967376    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="d549e05f-a5a1-478c-bdb7-18b03af91b72" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967384    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="5af5bf28-fe61-4653-b16a-8640b3156119" containerName="main"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967395    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="38eea1db-b2e8-4364-a7a9-b7bb655a20f1" containerName="main"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967403    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="122dd6bc-5f41-4ad1-bd2c-568c4a79baa7" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967412    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="4387fbe5-ef9e-4b48-9251-8181f1f941f9" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967422    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="df7bec60-2e19-4184-9432-a08efd6ea203" containerName="main"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967431    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="3fee816e-2cfa-4ba1-a6b1-a36260877599" containerName="maas-api"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967440    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="76814bbc-dc45-440a-947c-1f4979ad86de" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967450    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="298f7d43-f4d7-4108-b8a1-8df13e36ccc9" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967459    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="82ef457f-b231-432d-9c89-52e22180cea2" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967469    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb50b6fe-df28-4b1d-8a77-f7cdecf833c7" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967478    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ceb5aa1-a956-4442-b710-36453c7a1e79" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967485    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e886cd0-4ff5-4364-8a6b-3124b16e9cdc" containerName="istio-proxy"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967498    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="933b980b-58f5-44d0-b7b6-9f3fd58bcb63" containerName="main"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967509    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="b380cf76-c519-4719-84fa-5c3025ea84f1" containerName="maas-api"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967516    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="609e8c8e-04cc-4f27-b80a-0ab170d8ab57" containerName="maas-api"
Nov 03 21:12:25.967797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.967526    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3ba8495-3500-4f1c-b81b-8fd8dd56ad3e" containerName="main"
Nov 03 21:12:25.971120 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.971093    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-6e0ed0"
Nov 03 21:12:25.978356 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:25.978234    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz2lz\" (UniqueName: \"kubernetes.io/projected/8a85401b-cd66-4c87-b53f-db45688fa850-kube-api-access-gz2lz\") pod \"test-curl-7364-6e0ed0\" (UID: \"8a85401b-cd66-4c87-b53f-db45688fa850\") " pod="opendatahub/test-curl-7364-6e0ed0"
Nov 03 21:12:26.024296 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:26.024262    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-6e0ed0"]
Nov 03 21:12:26.071277 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:26.071248    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-nrr89\""
Nov 03 21:12:26.080201 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:26.080168    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz2lz\" (UniqueName: \"kubernetes.io/projected/8a85401b-cd66-4c87-b53f-db45688fa850-kube-api-access-gz2lz\") pod \"test-curl-7364-6e0ed0\" (UID: \"8a85401b-cd66-4c87-b53f-db45688fa850\") " pod="opendatahub/test-curl-7364-6e0ed0"
Nov 03 21:12:26.141428 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:26.141395    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz2lz\" (UniqueName: \"kubernetes.io/projected/8a85401b-cd66-4c87-b53f-db45688fa850-kube-api-access-gz2lz\") pod \"test-curl-7364-6e0ed0\" (UID: \"8a85401b-cd66-4c87-b53f-db45688fa850\") " pod="opendatahub/test-curl-7364-6e0ed0"
Nov 03 21:12:26.284457 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:26.284376    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-6e0ed0"
Nov 03 21:12:26.690174 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:26.690137    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-6e0ed0"]
Nov 03 21:12:26.692350 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:12:26.692319    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a85401b_cd66_4c87_b53f_db45688fa850.slice/crio-6b4b066e09ae752dbd32401ad8dd0fe1a5a1eff01a00f340203639246560ebb2 WatchSource:0}: Error finding container 6b4b066e09ae752dbd32401ad8dd0fe1a5a1eff01a00f340203639246560ebb2: Status 404 returned error can't find the container with id 6b4b066e09ae752dbd32401ad8dd0fe1a5a1eff01a00f340203639246560ebb2
Nov 03 21:12:26.805877 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:26.805832    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-6e0ed0" event={"ID":"8a85401b-cd66-4c87-b53f-db45688fa850","Type":"ContainerStarted","Data":"77d81325d6fdaff996f62c629c76a5e673ba90477baa31ca68dbe5c036885b8d"}
Nov 03 21:12:26.806059 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:26.805892    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-6e0ed0" event={"ID":"8a85401b-cd66-4c87-b53f-db45688fa850","Type":"ContainerStarted","Data":"6b4b066e09ae752dbd32401ad8dd0fe1a5a1eff01a00f340203639246560ebb2"}
Nov 03 21:12:26.919421 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:26.919302    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7364-6e0ed0" podStartSLOduration=1.919285347 podStartE2EDuration="1.919285347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:12:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:12:26.91884869 +0000 UTC m=+1949.463555062" watchObservedRunningTime="2026-11-03 21:12:26.919285347 +0000 UTC m=+1949.463991716"
Nov 03 21:12:28.470933 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:28.470893    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7364-6e0ed0"]
Nov 03 21:12:28.473664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:28.473621    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7364-6e0ed0"]
Nov 03 21:12:28.635759 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:28.635725    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7364-9de343"]
Nov 03 21:12:28.636211 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:28.636197    4993 cpu_manager.go:414] "containerMap: removing container" podUID="8a85401b-cd66-4c87-b53f-db45688fa850" containerName="test-curl-7364-6e0ed0"
Nov 03 21:12:28.636251 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:28.636213    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a85401b-cd66-4c87-b53f-db45688fa850" containerName="test-curl-7364-6e0ed0"
Nov 03 21:12:28.636299 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:28.636290    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a85401b-cd66-4c87-b53f-db45688fa850" containerName="test-curl-7364-6e0ed0"
Nov 03 21:12:28.639527 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:28.639511    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-9de343"
Nov 03 21:12:28.695474 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:28.695442    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-9de343"]
Nov 03 21:12:28.714040 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:28.714001    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg5cw\" (UniqueName: \"kubernetes.io/projected/eb668246-3311-45ab-af0b-374d49f8d620-kube-api-access-dg5cw\") pod \"test-curl-7364-9de343\" (UID: \"eb668246-3311-45ab-af0b-374d49f8d620\") " pod="opendatahub/test-curl-7364-9de343"
Nov 03 21:12:28.815657 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:28.815533    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg5cw\" (UniqueName: \"kubernetes.io/projected/eb668246-3311-45ab-af0b-374d49f8d620-kube-api-access-dg5cw\") pod \"test-curl-7364-9de343\" (UID: \"eb668246-3311-45ab-af0b-374d49f8d620\") " pod="opendatahub/test-curl-7364-9de343"
Nov 03 21:12:28.818654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:28.818608    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7364-6e0ed0" podUID="8a85401b-cd66-4c87-b53f-db45688fa850" containerName="test-curl-7364-6e0ed0" containerID="cri-o://77d81325d6fdaff996f62c629c76a5e673ba90477baa31ca68dbe5c036885b8d" gracePeriod=2
Nov 03 21:12:28.869355 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:28.869323    4993 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7364-6e0ed0\" is forbidden: User \"system:node:ip-10-0-142-223.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-223.us-west-1.compute.internal' and this object" podUID="8a85401b-cd66-4c87-b53f-db45688fa850" pod="opendatahub/test-curl-7364-6e0ed0"
Nov 03 21:12:28.876129 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:28.876107    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg5cw\" (UniqueName: \"kubernetes.io/projected/eb668246-3311-45ab-af0b-374d49f8d620-kube-api-access-dg5cw\") pod \"test-curl-7364-9de343\" (UID: \"eb668246-3311-45ab-af0b-374d49f8d620\") " pod="opendatahub/test-curl-7364-9de343"
Nov 03 21:12:28.951110 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:28.951086    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-9de343"
Nov 03 21:12:29.323895 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:29.323869    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-9de343"]
Nov 03 21:12:29.324920 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:12:29.324892    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb668246_3311_45ab_af0b_374d49f8d620.slice/crio-6f964117e83437d43b52a992d71c2126d5e2707b97b83532ec37d7c41cf14839 WatchSource:0}: Error finding container 6f964117e83437d43b52a992d71c2126d5e2707b97b83532ec37d7c41cf14839: Status 404 returned error can't find the container with id 6f964117e83437d43b52a992d71c2126d5e2707b97b83532ec37d7c41cf14839
Nov 03 21:12:29.870356 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:29.870319    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-9de343" event={"ID":"eb668246-3311-45ab-af0b-374d49f8d620","Type":"ContainerStarted","Data":"4a88b9caabcc2adecc8af415aa1856d362031a6de48c50755c589a50d1bf380c"}
Nov 03 21:12:29.870924 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:29.870902    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-9de343" event={"ID":"eb668246-3311-45ab-af0b-374d49f8d620","Type":"ContainerStarted","Data":"6f964117e83437d43b52a992d71c2126d5e2707b97b83532ec37d7c41cf14839"}
Nov 03 21:12:29.986067 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:29.986012    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7364-9de343" podStartSLOduration=1.985996948 podStartE2EDuration="1.985996948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:12:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:12:29.984211406 +0000 UTC m=+1952.528917777" watchObservedRunningTime="2026-11-03 21:12:29.985996948 +0000 UTC m=+1952.530703323"
Nov 03 21:12:30.877429 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:30.877402    4993 generic.go:356] "Generic (PLEG): container finished" podID="8a85401b-cd66-4c87-b53f-db45688fa850" containerID="77d81325d6fdaff996f62c629c76a5e673ba90477baa31ca68dbe5c036885b8d" exitCode=137
Nov 03 21:12:31.059696 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.059672    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-6e0ed0"
Nov 03 21:12:31.155119 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.155037    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a85401b-cd66-4c87-b53f-db45688fa850-kube-api-access-gz2lz\" (UniqueName: \"kubernetes.io/projected/8a85401b-cd66-4c87-b53f-db45688fa850-kube-api-access-gz2lz\") pod \"8a85401b-cd66-4c87-b53f-db45688fa850\" (UID: \"8a85401b-cd66-4c87-b53f-db45688fa850\") "
Nov 03 21:12:31.157195 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.157165    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a85401b-cd66-4c87-b53f-db45688fa850-kube-api-access-gz2lz" pod "8a85401b-cd66-4c87-b53f-db45688fa850" (UID: "8a85401b-cd66-4c87-b53f-db45688fa850"). InnerVolumeSpecName "kube-api-access-gz2lz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:12:31.256734 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.256698    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gz2lz\" (UniqueName: \"kubernetes.io/projected/8a85401b-cd66-4c87-b53f-db45688fa850-kube-api-access-gz2lz\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:31.578029 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.577991    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a85401b-cd66-4c87-b53f-db45688fa850" path="/var/lib/kubelet/pods/8a85401b-cd66-4c87-b53f-db45688fa850/volumes"
Nov 03 21:12:31.578402 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.578386    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7364-9de343"]
Nov 03 21:12:31.578497 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.578424    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7364-9de343"]
Nov 03 21:12:31.670835 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:31.670800    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:12:31.733404 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.733364    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7364-2e0a11"]
Nov 03 21:12:31.733913 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.733893    4993 cpu_manager.go:414] "containerMap: removing container" podUID="eb668246-3311-45ab-af0b-374d49f8d620" containerName="test-curl-7364-9de343"
Nov 03 21:12:31.734007 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.733915    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb668246-3311-45ab-af0b-374d49f8d620" containerName="test-curl-7364-9de343"
Nov 03 21:12:31.734089 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.734074    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb668246-3311-45ab-af0b-374d49f8d620" containerName="test-curl-7364-9de343"
Nov 03 21:12:31.737386 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.737366    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-2e0a11"
Nov 03 21:12:31.791883 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.791853    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-2e0a11"]
Nov 03 21:12:31.864436 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.864342    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkjc6\" (UniqueName: \"kubernetes.io/projected/0dc215fd-4f69-49f8-8381-b372ebcd3bb1-kube-api-access-mkjc6\") pod \"test-curl-7364-2e0a11\" (UID: \"0dc215fd-4f69-49f8-8381-b372ebcd3bb1\") " pod="opendatahub/test-curl-7364-2e0a11"
Nov 03 21:12:31.884615 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.884588    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-6e0ed0"
Nov 03 21:12:31.885053 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.884642    4993 scope.go:122] "RemoveContainer" containerID="77d81325d6fdaff996f62c629c76a5e673ba90477baa31ca68dbe5c036885b8d"
Nov 03 21:12:31.885053 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.884680    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7364-9de343" podUID="eb668246-3311-45ab-af0b-374d49f8d620" containerName="test-curl-7364-9de343" containerID="cri-o://4a88b9caabcc2adecc8af415aa1856d362031a6de48c50755c589a50d1bf380c" gracePeriod=2
Nov 03 21:12:31.935621 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:31.935591    4993 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7364-9de343\" is forbidden: User \"system:node:ip-10-0-142-223.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-223.us-west-1.compute.internal' and this object" podUID="eb668246-3311-45ab-af0b-374d49f8d620" pod="opendatahub/test-curl-7364-9de343"
Nov 03 21:12:31.966054 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:31.966026    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkjc6\" (UniqueName: \"kubernetes.io/projected/0dc215fd-4f69-49f8-8381-b372ebcd3bb1-kube-api-access-mkjc6\") pod \"test-curl-7364-2e0a11\" (UID: \"0dc215fd-4f69-49f8-8381-b372ebcd3bb1\") " pod="opendatahub/test-curl-7364-2e0a11"
Nov 03 21:12:31.987345 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:31.987315    4993 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7364-9de343\" is forbidden: User \"system:node:ip-10-0-142-223.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-223.us-west-1.compute.internal' and this object" podUID="eb668246-3311-45ab-af0b-374d49f8d620" pod="opendatahub/test-curl-7364-9de343"
Nov 03 21:12:32.026349 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:32.026323    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkjc6\" (UniqueName: \"kubernetes.io/projected/0dc215fd-4f69-49f8-8381-b372ebcd3bb1-kube-api-access-mkjc6\") pod \"test-curl-7364-2e0a11\" (UID: \"0dc215fd-4f69-49f8-8381-b372ebcd3bb1\") " pod="opendatahub/test-curl-7364-2e0a11"
Nov 03 21:12:32.050035 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:32.050006    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-2e0a11"
Nov 03 21:12:32.425736 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:32.425712    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-2e0a11"]
Nov 03 21:12:32.428113 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:12:32.428081    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0dc215fd_4f69_49f8_8381_b372ebcd3bb1.slice/crio-6ac945f2e7f6866c5130d4a1357bbe8ebfb511bb77c488c6ece903ee0386805b WatchSource:0}: Error finding container 6ac945f2e7f6866c5130d4a1357bbe8ebfb511bb77c488c6ece903ee0386805b: Status 404 returned error can't find the container with id 6ac945f2e7f6866c5130d4a1357bbe8ebfb511bb77c488c6ece903ee0386805b
Nov 03 21:12:32.892610 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:32.892569    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-2e0a11" event={"ID":"0dc215fd-4f69-49f8-8381-b372ebcd3bb1","Type":"ContainerStarted","Data":"e625cd7537425ba4f6acc4e3c8091de72d80de06f57a28143bd7e5e164680eed"}
Nov 03 21:12:32.893085 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:32.892650    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-2e0a11" event={"ID":"0dc215fd-4f69-49f8-8381-b372ebcd3bb1","Type":"ContainerStarted","Data":"6ac945f2e7f6866c5130d4a1357bbe8ebfb511bb77c488c6ece903ee0386805b"}
Nov 03 21:12:32.943485 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:32.943444    4993 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7364-9de343\" is forbidden: User \"system:node:ip-10-0-142-223.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-223.us-west-1.compute.internal' and this object" podUID="eb668246-3311-45ab-af0b-374d49f8d620" pod="opendatahub/test-curl-7364-9de343"
Nov 03 21:12:33.060042 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:33.059978    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7364-2e0a11" podStartSLOduration=2.059964855 podStartE2EDuration="2.059964855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:12:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:12:33.058787822 +0000 UTC m=+1955.603494194" watchObservedRunningTime="2026-11-03 21:12:33.059964855 +0000 UTC m=+1955.604671224"
Nov 03 21:12:34.100341 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.100309    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-9de343"
Nov 03 21:12:34.191580 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.191487    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb668246-3311-45ab-af0b-374d49f8d620-kube-api-access-dg5cw\" (UniqueName: \"kubernetes.io/projected/eb668246-3311-45ab-af0b-374d49f8d620-kube-api-access-dg5cw\") pod \"eb668246-3311-45ab-af0b-374d49f8d620\" (UID: \"eb668246-3311-45ab-af0b-374d49f8d620\") "
Nov 03 21:12:34.193775 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.193751    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb668246-3311-45ab-af0b-374d49f8d620-kube-api-access-dg5cw" pod "eb668246-3311-45ab-af0b-374d49f8d620" (UID: "eb668246-3311-45ab-af0b-374d49f8d620"). InnerVolumeSpecName "kube-api-access-dg5cw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:12:34.293311 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.293274    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dg5cw\" (UniqueName: \"kubernetes.io/projected/eb668246-3311-45ab-af0b-374d49f8d620-kube-api-access-dg5cw\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:34.519975 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.519935    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7364-2e0a11"]
Nov 03 21:12:34.523554 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.523514    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7364-2e0a11"]
Nov 03 21:12:34.703089 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.703050    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7364-d5cebd"]
Nov 03 21:12:34.703540 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.703525    4993 cpu_manager.go:414] "containerMap: removing container" podUID="0dc215fd-4f69-49f8-8381-b372ebcd3bb1" containerName="test-curl-7364-2e0a11"
Nov 03 21:12:34.703599 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.703543    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0dc215fd-4f69-49f8-8381-b372ebcd3bb1" containerName="test-curl-7364-2e0a11"
Nov 03 21:12:34.703683 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.703673    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="0dc215fd-4f69-49f8-8381-b372ebcd3bb1" containerName="test-curl-7364-2e0a11"
Nov 03 21:12:34.706899 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.706883    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-d5cebd"
Nov 03 21:12:34.776108 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.776025    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-d5cebd"]
Nov 03 21:12:34.799544 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.799509    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l97z8\" (UniqueName: \"kubernetes.io/projected/93aa090e-efe6-4953-8368-cb2b7bcdc394-kube-api-access-l97z8\") pod \"test-curl-7364-d5cebd\" (UID: \"93aa090e-efe6-4953-8368-cb2b7bcdc394\") " pod="opendatahub/test-curl-7364-d5cebd"
Nov 03 21:12:34.901472 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.901433    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l97z8\" (UniqueName: \"kubernetes.io/projected/93aa090e-efe6-4953-8368-cb2b7bcdc394-kube-api-access-l97z8\") pod \"test-curl-7364-d5cebd\" (UID: \"93aa090e-efe6-4953-8368-cb2b7bcdc394\") " pod="opendatahub/test-curl-7364-d5cebd"
Nov 03 21:12:34.907391 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.907361    4993 generic.go:356] "Generic (PLEG): container finished" podID="eb668246-3311-45ab-af0b-374d49f8d620" containerID="4a88b9caabcc2adecc8af415aa1856d362031a6de48c50755c589a50d1bf380c" exitCode=137
Nov 03 21:12:34.907561 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.907409    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-9de343"
Nov 03 21:12:34.907561 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.907462    4993 scope.go:122] "RemoveContainer" containerID="4a88b9caabcc2adecc8af415aa1856d362031a6de48c50755c589a50d1bf380c"
Nov 03 21:12:34.907695 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.907591    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7364-2e0a11" podUID="0dc215fd-4f69-49f8-8381-b372ebcd3bb1" containerName="test-curl-7364-2e0a11" containerID="cri-o://e625cd7537425ba4f6acc4e3c8091de72d80de06f57a28143bd7e5e164680eed" gracePeriod=2
Nov 03 21:12:34.916963 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.916945    4993 scope.go:122] "RemoveContainer" containerID="4a88b9caabcc2adecc8af415aa1856d362031a6de48c50755c589a50d1bf380c"
Nov 03 21:12:34.917253 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:34.917228    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a88b9caabcc2adecc8af415aa1856d362031a6de48c50755c589a50d1bf380c\": container with ID starting with 4a88b9caabcc2adecc8af415aa1856d362031a6de48c50755c589a50d1bf380c not found: ID does not exist" containerID="4a88b9caabcc2adecc8af415aa1856d362031a6de48c50755c589a50d1bf380c"
Nov 03 21:12:34.917310 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.917265    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a88b9caabcc2adecc8af415aa1856d362031a6de48c50755c589a50d1bf380c"} err="failed to get container status \"4a88b9caabcc2adecc8af415aa1856d362031a6de48c50755c589a50d1bf380c\": rpc error: code = NotFound desc = could not find container \"4a88b9caabcc2adecc8af415aa1856d362031a6de48c50755c589a50d1bf380c\": container with ID starting with 4a88b9caabcc2adecc8af415aa1856d362031a6de48c50755c589a50d1bf380c not found: ID does not exist"
Nov 03 21:12:34.958843 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:34.958806    4993 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7364-2e0a11\" is forbidden: User \"system:node:ip-10-0-142-223.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-223.us-west-1.compute.internal' and this object" podUID="0dc215fd-4f69-49f8-8381-b372ebcd3bb1" pod="opendatahub/test-curl-7364-2e0a11"
Nov 03 21:12:34.962216 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:34.962196    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l97z8\" (UniqueName: \"kubernetes.io/projected/93aa090e-efe6-4953-8368-cb2b7bcdc394-kube-api-access-l97z8\") pod \"test-curl-7364-d5cebd\" (UID: \"93aa090e-efe6-4953-8368-cb2b7bcdc394\") " pod="opendatahub/test-curl-7364-d5cebd"
Nov 03 21:12:35.009729 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:35.009683    4993 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7364-9de343\" is forbidden: User \"system:node:ip-10-0-142-223.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-223.us-west-1.compute.internal' and this object" podUID="eb668246-3311-45ab-af0b-374d49f8d620" pod="opendatahub/test-curl-7364-9de343"
Nov 03 21:12:35.018879 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:35.018851    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-d5cebd"
Nov 03 21:12:35.060655 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:35.060588    4993 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7364-2e0a11\" is forbidden: User \"system:node:ip-10-0-142-223.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-223.us-west-1.compute.internal' and this object" podUID="0dc215fd-4f69-49f8-8381-b372ebcd3bb1" pod="opendatahub/test-curl-7364-2e0a11"
Nov 03 21:12:35.404083 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:35.404053    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7364-d5cebd"]
Nov 03 21:12:35.406245 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:12:35.406216    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93aa090e_efe6_4953_8368_cb2b7bcdc394.slice/crio-248667c0b2f581fd8c998b9726098de1c983ce351c342492d0b9d486f67bf3dd WatchSource:0}: Error finding container 248667c0b2f581fd8c998b9726098de1c983ce351c342492d0b9d486f67bf3dd: Status 404 returned error can't find the container with id 248667c0b2f581fd8c998b9726098de1c983ce351c342492d0b9d486f67bf3dd
Nov 03 21:12:35.576603 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:35.576572    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb668246-3311-45ab-af0b-374d49f8d620" path="/var/lib/kubelet/pods/eb668246-3311-45ab-af0b-374d49f8d620/volumes"
Nov 03 21:12:35.915423 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:35.915384    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-d5cebd" event={"ID":"93aa090e-efe6-4953-8368-cb2b7bcdc394","Type":"ContainerStarted","Data":"407daf5c8e35c3071532d2fcdcb2462d3a956182f963cc60791d7c4c3b288a9f"}
Nov 03 21:12:35.915423 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:35.915436    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7364-d5cebd" event={"ID":"93aa090e-efe6-4953-8368-cb2b7bcdc394","Type":"ContainerStarted","Data":"248667c0b2f581fd8c998b9726098de1c983ce351c342492d0b9d486f67bf3dd"}
Nov 03 21:12:36.033844 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:36.033796    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7364-d5cebd" podStartSLOduration=2.03378156 podStartE2EDuration="2.03378156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:12:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:12:36.033103989 +0000 UTC m=+1958.577810359" watchObservedRunningTime="2026-11-03 21:12:36.03378156 +0000 UTC m=+1958.578487929"
Nov 03 21:12:37.192681 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:37.192657    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-2e0a11"
Nov 03 21:12:37.333850 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:37.333822    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0dc215fd-4f69-49f8-8381-b372ebcd3bb1-kube-api-access-mkjc6\" (UniqueName: \"kubernetes.io/projected/0dc215fd-4f69-49f8-8381-b372ebcd3bb1-kube-api-access-mkjc6\") pod \"0dc215fd-4f69-49f8-8381-b372ebcd3bb1\" (UID: \"0dc215fd-4f69-49f8-8381-b372ebcd3bb1\") "
Nov 03 21:12:37.335967 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:37.335943    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dc215fd-4f69-49f8-8381-b372ebcd3bb1-kube-api-access-mkjc6" pod "0dc215fd-4f69-49f8-8381-b372ebcd3bb1" (UID: "0dc215fd-4f69-49f8-8381-b372ebcd3bb1"). InnerVolumeSpecName "kube-api-access-mkjc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:12:37.435053 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:37.434991    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkjc6\" (UniqueName: \"kubernetes.io/projected/0dc215fd-4f69-49f8-8381-b372ebcd3bb1-kube-api-access-mkjc6\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:37.585845 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:37.585814    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0dc215fd-4f69-49f8-8381-b372ebcd3bb1" path="/var/lib/kubelet/pods/0dc215fd-4f69-49f8-8381-b372ebcd3bb1/volumes"
Nov 03 21:12:37.595196 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:37.595162    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7364-d5cebd"]
Nov 03 21:12:37.597622 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:37.597595    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7364-d5cebd"]
Nov 03 21:12:37.929448 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:37.929416    4993 generic.go:356] "Generic (PLEG): container finished" podID="0dc215fd-4f69-49f8-8381-b372ebcd3bb1" containerID="e625cd7537425ba4f6acc4e3c8091de72d80de06f57a28143bd7e5e164680eed" exitCode=137
Nov 03 21:12:37.929657 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:37.929486    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-2e0a11"
Nov 03 21:12:37.929657 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:37.929526    4993 scope.go:122] "RemoveContainer" containerID="e625cd7537425ba4f6acc4e3c8091de72d80de06f57a28143bd7e5e164680eed"
Nov 03 21:12:37.929799 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:37.929780    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7364-d5cebd" podUID="93aa090e-efe6-4953-8368-cb2b7bcdc394" containerName="test-curl-7364-d5cebd" containerID="cri-o://407daf5c8e35c3071532d2fcdcb2462d3a956182f963cc60791d7c4c3b288a9f" gracePeriod=2
Nov 03 21:12:37.939491 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:37.939467    4993 scope.go:122] "RemoveContainer" containerID="e625cd7537425ba4f6acc4e3c8091de72d80de06f57a28143bd7e5e164680eed"
Nov 03 21:12:37.939831 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:37.939811    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e625cd7537425ba4f6acc4e3c8091de72d80de06f57a28143bd7e5e164680eed\": container with ID starting with e625cd7537425ba4f6acc4e3c8091de72d80de06f57a28143bd7e5e164680eed not found: ID does not exist" containerID="e625cd7537425ba4f6acc4e3c8091de72d80de06f57a28143bd7e5e164680eed"
Nov 03 21:12:37.939912 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:37.939840    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e625cd7537425ba4f6acc4e3c8091de72d80de06f57a28143bd7e5e164680eed"} err="failed to get container status \"e625cd7537425ba4f6acc4e3c8091de72d80de06f57a28143bd7e5e164680eed\": rpc error: code = NotFound desc = could not find container \"e625cd7537425ba4f6acc4e3c8091de72d80de06f57a28143bd7e5e164680eed\": container with ID starting with e625cd7537425ba4f6acc4e3c8091de72d80de06f57a28143bd7e5e164680eed not found: ID does not exist"
Nov 03 21:12:37.980732 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:37.980698    4993 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7364-d5cebd\" is forbidden: User \"system:node:ip-10-0-142-223.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-223.us-west-1.compute.internal' and this object" podUID="93aa090e-efe6-4953-8368-cb2b7bcdc394" pod="opendatahub/test-curl-7364-d5cebd"
Nov 03 21:12:38.031795 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:38.031755    4993 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7364-d5cebd\" is forbidden: User \"system:node:ip-10-0-142-223.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-223.us-west-1.compute.internal' and this object" podUID="93aa090e-efe6-4953-8368-cb2b7bcdc394" pod="opendatahub/test-curl-7364-d5cebd"
Nov 03 21:12:40.216270 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:40.216244    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-d5cebd"
Nov 03 21:12:40.264761 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:40.264731    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93aa090e-efe6-4953-8368-cb2b7bcdc394-kube-api-access-l97z8\" (UniqueName: \"kubernetes.io/projected/93aa090e-efe6-4953-8368-cb2b7bcdc394-kube-api-access-l97z8\") pod \"93aa090e-efe6-4953-8368-cb2b7bcdc394\" (UID: \"93aa090e-efe6-4953-8368-cb2b7bcdc394\") "
Nov 03 21:12:40.266984 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:40.266955    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93aa090e-efe6-4953-8368-cb2b7bcdc394-kube-api-access-l97z8" pod "93aa090e-efe6-4953-8368-cb2b7bcdc394" (UID: "93aa090e-efe6-4953-8368-cb2b7bcdc394"). InnerVolumeSpecName "kube-api-access-l97z8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:12:40.366649 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:40.366347    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l97z8\" (UniqueName: \"kubernetes.io/projected/93aa090e-efe6-4953-8368-cb2b7bcdc394-kube-api-access-l97z8\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:12:40.950832 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:40.950800    4993 generic.go:356] "Generic (PLEG): container finished" podID="93aa090e-efe6-4953-8368-cb2b7bcdc394" containerID="407daf5c8e35c3071532d2fcdcb2462d3a956182f963cc60791d7c4c3b288a9f" exitCode=137
Nov 03 21:12:40.951002 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:40.950839    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7364-d5cebd"
Nov 03 21:12:40.951002 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:40.950912    4993 scope.go:122] "RemoveContainer" containerID="407daf5c8e35c3071532d2fcdcb2462d3a956182f963cc60791d7c4c3b288a9f"
Nov 03 21:12:40.961324 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:40.961300    4993 scope.go:122] "RemoveContainer" containerID="407daf5c8e35c3071532d2fcdcb2462d3a956182f963cc60791d7c4c3b288a9f"
Nov 03 21:12:40.961576 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:40.961557    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"407daf5c8e35c3071532d2fcdcb2462d3a956182f963cc60791d7c4c3b288a9f\": container with ID starting with 407daf5c8e35c3071532d2fcdcb2462d3a956182f963cc60791d7c4c3b288a9f not found: ID does not exist" containerID="407daf5c8e35c3071532d2fcdcb2462d3a956182f963cc60791d7c4c3b288a9f"
Nov 03 21:12:40.961656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:40.961586    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407daf5c8e35c3071532d2fcdcb2462d3a956182f963cc60791d7c4c3b288a9f"} err="failed to get container status \"407daf5c8e35c3071532d2fcdcb2462d3a956182f963cc60791d7c4c3b288a9f\": rpc error: code = NotFound desc = could not find container \"407daf5c8e35c3071532d2fcdcb2462d3a956182f963cc60791d7c4c3b288a9f\": container with ID starting with 407daf5c8e35c3071532d2fcdcb2462d3a956182f963cc60791d7c4c3b288a9f not found: ID does not exist"
Nov 03 21:12:41.011305 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:12:41.011266    4993 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7364-d5cebd\" is forbidden: User \"system:node:ip-10-0-142-223.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-223.us-west-1.compute.internal' and this object" podUID="93aa090e-efe6-4953-8368-cb2b7bcdc394" pod="opendatahub/test-curl-7364-d5cebd"
Nov 03 21:12:41.579203 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:41.579160    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93aa090e-efe6-4953-8368-cb2b7bcdc394" path="/var/lib/kubelet/pods/93aa090e-efe6-4953-8368-cb2b7bcdc394/volumes"
Nov 03 21:12:45.648893 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:12:45.648864    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"]
Nov 03 21:13:00.669847 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.669783    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" containerID="cri-o://a53e586a99392a3d95ee9c5c59b8043638992da9fdf8289716db416934c7059f" gracePeriod=45
Nov 03 21:13:00.925682 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.925612    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 21:13:00.956246 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.956210    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c0c71be3-4788-4e30-8bfb-905e950a6d0a-tls-certs\" (UniqueName: \"kubernetes.io/secret/c0c71be3-4788-4e30-8bfb-905e950a6d0a-tls-certs\") pod \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") "
Nov 03 21:13:00.956246 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.956247    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-dshm\") pod \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") "
Nov 03 21:13:00.956464 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.956282    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-model-cache\") pod \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") "
Nov 03 21:13:00.956464 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.956448    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0c71be3-4788-4e30-8bfb-905e950a6d0a-kube-api-access-7rxl2\" (UniqueName: \"kubernetes.io/projected/c0c71be3-4788-4e30-8bfb-905e950a6d0a-kube-api-access-7rxl2\") pod \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") "
Nov 03 21:13:00.956584 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.956492    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-tmp-dir\") pod \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") "
Nov 03 21:13:00.956584 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.956527    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-home\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-home\") pod \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\" (UID: \"c0c71be3-4788-4e30-8bfb-905e950a6d0a\") "
Nov 03 21:13:00.956704 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.956601    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-model-cache" pod "c0c71be3-4788-4e30-8bfb-905e950a6d0a" (UID: "c0c71be3-4788-4e30-8bfb-905e950a6d0a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:00.956806 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.956774    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-tmp-dir" pod "c0c71be3-4788-4e30-8bfb-905e950a6d0a" (UID: "c0c71be3-4788-4e30-8bfb-905e950a6d0a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:00.956933 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.956911    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-home" pod "c0c71be3-4788-4e30-8bfb-905e950a6d0a" (UID: "c0c71be3-4788-4e30-8bfb-905e950a6d0a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:00.957272 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.957244    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:00.957406 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.957274    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:00.957406 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.957289    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:00.958993 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.958969    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0c71be3-4788-4e30-8bfb-905e950a6d0a-tls-certs" pod "c0c71be3-4788-4e30-8bfb-905e950a6d0a" (UID: "c0c71be3-4788-4e30-8bfb-905e950a6d0a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:13:00.959306 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.959275    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-dshm" pod "c0c71be3-4788-4e30-8bfb-905e950a6d0a" (UID: "c0c71be3-4788-4e30-8bfb-905e950a6d0a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:00.959472 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:00.959452    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0c71be3-4788-4e30-8bfb-905e950a6d0a-kube-api-access-7rxl2" pod "c0c71be3-4788-4e30-8bfb-905e950a6d0a" (UID: "c0c71be3-4788-4e30-8bfb-905e950a6d0a"). InnerVolumeSpecName "kube-api-access-7rxl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:01.062664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:01.062609    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rxl2\" (UniqueName: \"kubernetes.io/projected/c0c71be3-4788-4e30-8bfb-905e950a6d0a-kube-api-access-7rxl2\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:01.062664 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:01.062664    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0c71be3-4788-4e30-8bfb-905e950a6d0a-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:01.062921 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:01.062681    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0c71be3-4788-4e30-8bfb-905e950a6d0a-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:01.099687 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:01.099601    4993 generic.go:356] "Generic (PLEG): container finished" podID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerID="a53e586a99392a3d95ee9c5c59b8043638992da9fdf8289716db416934c7059f" exitCode=0
Nov 03 21:13:01.099871 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:01.099709    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"
Nov 03 21:13:01.099871 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:01.099699    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" event={"ID":"c0c71be3-4788-4e30-8bfb-905e950a6d0a","Type":"ContainerDied","Data":"a53e586a99392a3d95ee9c5c59b8043638992da9fdf8289716db416934c7059f"}
Nov 03 21:13:01.100020 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:01.099872    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" event={"ID":"c0c71be3-4788-4e30-8bfb-905e950a6d0a","Type":"ContainerDied","Data":"0fb2f7fe219ee9ffbbe619f1480d72936453f5897f0f2390bafe72000fe36a5b"}
Nov 03 21:13:01.100020 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:01.099899    4993 scope.go:122] "RemoveContainer" containerID="a53e586a99392a3d95ee9c5c59b8043638992da9fdf8289716db416934c7059f"
Nov 03 21:13:01.125776 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:01.125748    4993 scope.go:122] "RemoveContainer" containerID="a53e586a99392a3d95ee9c5c59b8043638992da9fdf8289716db416934c7059f"
Nov 03 21:13:01.126163 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:13:01.126132    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a53e586a99392a3d95ee9c5c59b8043638992da9fdf8289716db416934c7059f\": container with ID starting with a53e586a99392a3d95ee9c5c59b8043638992da9fdf8289716db416934c7059f not found: ID does not exist" containerID="a53e586a99392a3d95ee9c5c59b8043638992da9fdf8289716db416934c7059f"
Nov 03 21:13:01.126268 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:01.126171    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a53e586a99392a3d95ee9c5c59b8043638992da9fdf8289716db416934c7059f"} err="failed to get container status \"a53e586a99392a3d95ee9c5c59b8043638992da9fdf8289716db416934c7059f\": rpc error: code = NotFound desc = could not find container \"a53e586a99392a3d95ee9c5c59b8043638992da9fdf8289716db416934c7059f\": container with ID starting with a53e586a99392a3d95ee9c5c59b8043638992da9fdf8289716db416934c7059f not found: ID does not exist"
Nov 03 21:13:01.289568 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:01.289532    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"]
Nov 03 21:13:01.291502 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:01.291473    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt"]
Nov 03 21:13:01.582153 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:01.582080    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" path="/var/lib/kubelet/pods/c0c71be3-4788-4e30-8bfb-905e950a6d0a/volumes"
Nov 03 21:13:01.673117 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:13:01.673074    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:13:01.845486 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:01.845401    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d01cef/simulator-w1-d01cef-kserve-6fc76fd5d9-wjklt" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:13:12.868857 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:12.868778    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"]
Nov 03 21:13:12.869315 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:12.869044    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m" podUID="529e3166-0d8e-40e1-93a7-a6b563ce5c8c" containerName="istio-proxy" containerID="cri-o://0562b913997b3c2ef80cc31808bce032050be81a3704bcecd3f6789395f1b200" gracePeriod=30
Nov 03 21:13:15.375861 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:15.375825    4993 patch_prober.go:29] interesting pod/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:13:15.376601 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:15.376564    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m" podUID="529e3166-0d8e-40e1-93a7-a6b563ce5c8c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:13:18.165534 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.165507    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 21:13:18.200654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.200332    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-envoy\") pod \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") "
Nov 03 21:13:18.200654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.200393    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-workload-certs\") pod \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") "
Nov 03 21:13:18.200654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.200438    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-token\" (UniqueName: \"kubernetes.io/projected/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-token\") pod \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") "
Nov 03 21:13:18.200654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.200511    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-workload-socket\") pod \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") "
Nov 03 21:13:18.200654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.200560    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-podinfo\") pod \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") "
Nov 03 21:13:18.200654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.200621    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-kube-api-access-mf9kt\" (UniqueName: \"kubernetes.io/projected/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-kube-api-access-mf9kt\") pod \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") "
Nov 03 21:13:18.201133 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.200683    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-credential-socket\") pod \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") "
Nov 03 21:13:18.201133 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.200794    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-data\") pod \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") "
Nov 03 21:13:18.201133 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.200859    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istiod-ca-cert\") pod \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\" (UID: \"529e3166-0d8e-40e1-93a7-a6b563ce5c8c\") "
Nov 03 21:13:18.213508 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.201791    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-workload-socket" pod "529e3166-0d8e-40e1-93a7-a6b563ce5c8c" (UID: "529e3166-0d8e-40e1-93a7-a6b563ce5c8c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:18.213508 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.201831    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istiod-ca-cert" pod "529e3166-0d8e-40e1-93a7-a6b563ce5c8c" (UID: "529e3166-0d8e-40e1-93a7-a6b563ce5c8c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:13:18.213508 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.205113    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-envoy" pod "529e3166-0d8e-40e1-93a7-a6b563ce5c8c" (UID: "529e3166-0d8e-40e1-93a7-a6b563ce5c8c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:18.213508 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.205537    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-podinfo" pod "529e3166-0d8e-40e1-93a7-a6b563ce5c8c" (UID: "529e3166-0d8e-40e1-93a7-a6b563ce5c8c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:13:18.213508 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.206405    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-data" pod "529e3166-0d8e-40e1-93a7-a6b563ce5c8c" (UID: "529e3166-0d8e-40e1-93a7-a6b563ce5c8c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:18.213508 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.206704    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-credential-socket" pod "529e3166-0d8e-40e1-93a7-a6b563ce5c8c" (UID: "529e3166-0d8e-40e1-93a7-a6b563ce5c8c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:18.213508 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.207044    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-workload-certs" pod "529e3166-0d8e-40e1-93a7-a6b563ce5c8c" (UID: "529e3166-0d8e-40e1-93a7-a6b563ce5c8c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:18.213508 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.211326    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-kube-api-access-mf9kt" pod "529e3166-0d8e-40e1-93a7-a6b563ce5c8c" (UID: "529e3166-0d8e-40e1-93a7-a6b563ce5c8c"). InnerVolumeSpecName "kube-api-access-mf9kt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:18.213508 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.211431    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-token" pod "529e3166-0d8e-40e1-93a7-a6b563ce5c8c" (UID: "529e3166-0d8e-40e1-93a7-a6b563ce5c8c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:18.225332 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.225290    4993 generic.go:356] "Generic (PLEG): container finished" podID="529e3166-0d8e-40e1-93a7-a6b563ce5c8c" containerID="0562b913997b3c2ef80cc31808bce032050be81a3704bcecd3f6789395f1b200" exitCode=0
Nov 03 21:13:18.225442 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.225338    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"
Nov 03 21:13:18.225442 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.225374    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m" event={"ID":"529e3166-0d8e-40e1-93a7-a6b563ce5c8c","Type":"ContainerDied","Data":"0562b913997b3c2ef80cc31808bce032050be81a3704bcecd3f6789395f1b200"}
Nov 03 21:13:18.225442 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.225440    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m" event={"ID":"529e3166-0d8e-40e1-93a7-a6b563ce5c8c","Type":"ContainerDied","Data":"60c2e91d1b1e5c0e03a87e2d7d4f3f5568eeb1ce3e691129f1b71bcd5ca46b34"}
Nov 03 21:13:18.225599 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.225464    4993 scope.go:122] "RemoveContainer" containerID="0562b913997b3c2ef80cc31808bce032050be81a3704bcecd3f6789395f1b200"
Nov 03 21:13:18.235646 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.235612    4993 scope.go:122] "RemoveContainer" containerID="0562b913997b3c2ef80cc31808bce032050be81a3704bcecd3f6789395f1b200"
Nov 03 21:13:18.235927 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:13:18.235905    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0562b913997b3c2ef80cc31808bce032050be81a3704bcecd3f6789395f1b200\": container with ID starting with 0562b913997b3c2ef80cc31808bce032050be81a3704bcecd3f6789395f1b200 not found: ID does not exist" containerID="0562b913997b3c2ef80cc31808bce032050be81a3704bcecd3f6789395f1b200"
Nov 03 21:13:18.235996 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.235940    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0562b913997b3c2ef80cc31808bce032050be81a3704bcecd3f6789395f1b200"} err="failed to get container status \"0562b913997b3c2ef80cc31808bce032050be81a3704bcecd3f6789395f1b200\": rpc error: code = NotFound desc = could not find container \"0562b913997b3c2ef80cc31808bce032050be81a3704bcecd3f6789395f1b200\": container with ID starting with 0562b913997b3c2ef80cc31808bce032050be81a3704bcecd3f6789395f1b200 not found: ID does not exist"
Nov 03 21:13:18.302777 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.302750    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:18.302777 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.302774    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:18.302918 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.302784    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf9kt\" (UniqueName: \"kubernetes.io/projected/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-kube-api-access-mf9kt\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:18.302918 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.302794    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:18.302918 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.302803    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:18.302918 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.302811    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:18.302918 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.302819    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:18.302918 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.302828    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:18.302918 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.302836    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/529e3166-0d8e-40e1-93a7-a6b563ce5c8c-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:18.504297 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.504263    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"]
Nov 03 21:13:18.508981 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:18.508956    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-6pk2m"]
Nov 03 21:13:19.577419 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:19.577383    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="529e3166-0d8e-40e1-93a7-a6b563ce5c8c" path="/var/lib/kubelet/pods/529e3166-0d8e-40e1-93a7-a6b563ce5c8c/volumes"
Nov 03 21:13:31.671265 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:13:31.671225    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:13:36.252269 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:36.252232    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"]
Nov 03 21:13:36.252706 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:36.252437    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg" podUID="dcdc5f37-f37d-43bc-8fea-ad101a050eec" containerName="maas-api" containerID="cri-o://b3b2cecabb8048e7170d8c895f0008dfe4276694ceb99c768a2c0a66970c9f79" gracePeriod=30
Nov 03 21:13:36.562670 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:36.562646    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 21:13:36.633454 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:36.633416    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dcdc5f37-f37d-43bc-8fea-ad101a050eec-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcdc5f37-f37d-43bc-8fea-ad101a050eec-openshift-service-ca\") pod \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") "
Nov 03 21:13:36.633661 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:36.633474    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-metrics-certs\") pod \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") "
Nov 03 21:13:36.633661 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:36.633590    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dcdc5f37-f37d-43bc-8fea-ad101a050eec-kube-api-access-92czx\" (UniqueName: \"kubernetes.io/projected/dcdc5f37-f37d-43bc-8fea-ad101a050eec-kube-api-access-92czx\") pod \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") "
Nov 03 21:13:36.633783 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:36.633670    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-maas-api-tls\") pod \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\" (UID: \"dcdc5f37-f37d-43bc-8fea-ad101a050eec\") "
Nov 03 21:13:36.634407 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:36.634360    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dcdc5f37-f37d-43bc-8fea-ad101a050eec-openshift-service-ca" pod "dcdc5f37-f37d-43bc-8fea-ad101a050eec" (UID: "dcdc5f37-f37d-43bc-8fea-ad101a050eec"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:13:36.637411 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:36.637381    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-metrics-certs" pod "dcdc5f37-f37d-43bc-8fea-ad101a050eec" (UID: "dcdc5f37-f37d-43bc-8fea-ad101a050eec"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:13:36.638516 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:36.638489    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcdc5f37-f37d-43bc-8fea-ad101a050eec-kube-api-access-92czx" pod "dcdc5f37-f37d-43bc-8fea-ad101a050eec" (UID: "dcdc5f37-f37d-43bc-8fea-ad101a050eec"). InnerVolumeSpecName "kube-api-access-92czx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:36.639681 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:36.639622    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-maas-api-tls" pod "dcdc5f37-f37d-43bc-8fea-ad101a050eec" (UID: "dcdc5f37-f37d-43bc-8fea-ad101a050eec"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:13:36.735523 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:36.735485    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92czx\" (UniqueName: \"kubernetes.io/projected/dcdc5f37-f37d-43bc-8fea-ad101a050eec-kube-api-access-92czx\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:36.735523 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:36.735516    4993 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-maas-api-tls\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:36.735523 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:36.735526    4993 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dcdc5f37-f37d-43bc-8fea-ad101a050eec-openshift-service-ca\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:36.735804 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:36.735535    4993 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcdc5f37-f37d-43bc-8fea-ad101a050eec-metrics-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:37.346450 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:37.346415    4993 generic.go:356] "Generic (PLEG): container finished" podID="dcdc5f37-f37d-43bc-8fea-ad101a050eec" containerID="b3b2cecabb8048e7170d8c895f0008dfe4276694ceb99c768a2c0a66970c9f79" exitCode=0
Nov 03 21:13:37.346959 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:37.346462    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"
Nov 03 21:13:37.346959 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:37.346500    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg" event={"ID":"dcdc5f37-f37d-43bc-8fea-ad101a050eec","Type":"ContainerDied","Data":"b3b2cecabb8048e7170d8c895f0008dfe4276694ceb99c768a2c0a66970c9f79"}
Nov 03 21:13:37.346959 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:37.346561    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg" event={"ID":"dcdc5f37-f37d-43bc-8fea-ad101a050eec","Type":"ContainerDied","Data":"a0f60d2f9177077d446b4699a9fc7dcb3c1129c850a106c4cd5ea5c0608fd864"}
Nov 03 21:13:37.346959 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:37.346589    4993 scope.go:122] "RemoveContainer" containerID="b3b2cecabb8048e7170d8c895f0008dfe4276694ceb99c768a2c0a66970c9f79"
Nov 03 21:13:37.356979 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:37.356959    4993 scope.go:122] "RemoveContainer" containerID="b3b2cecabb8048e7170d8c895f0008dfe4276694ceb99c768a2c0a66970c9f79"
Nov 03 21:13:37.357234 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:13:37.357215    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3b2cecabb8048e7170d8c895f0008dfe4276694ceb99c768a2c0a66970c9f79\": container with ID starting with b3b2cecabb8048e7170d8c895f0008dfe4276694ceb99c768a2c0a66970c9f79 not found: ID does not exist" containerID="b3b2cecabb8048e7170d8c895f0008dfe4276694ceb99c768a2c0a66970c9f79"
Nov 03 21:13:37.357280 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:37.357244    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3b2cecabb8048e7170d8c895f0008dfe4276694ceb99c768a2c0a66970c9f79"} err="failed to get container status \"b3b2cecabb8048e7170d8c895f0008dfe4276694ceb99c768a2c0a66970c9f79\": rpc error: code = NotFound desc = could not find container \"b3b2cecabb8048e7170d8c895f0008dfe4276694ceb99c768a2c0a66970c9f79\": container with ID starting with b3b2cecabb8048e7170d8c895f0008dfe4276694ceb99c768a2c0a66970c9f79 not found: ID does not exist"
Nov 03 21:13:37.516869 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:37.516832    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"]
Nov 03 21:13:37.551023 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:37.550996    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d01cef-64fffc44f9-mdmdg"]
Nov 03 21:13:37.579473 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:37.579438    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dcdc5f37-f37d-43bc-8fea-ad101a050eec" path="/var/lib/kubelet/pods/dcdc5f37-f37d-43bc-8fea-ad101a050eec/volumes"
Nov 03 21:13:46.041291 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:46.041253    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"]
Nov 03 21:13:46.041852 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:46.041458    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m" podUID="68cccb32-d2a6-48f7-98f6-f0f60aab5496" containerName="istio-proxy" containerID="cri-o://989be0d73dfc6cfdcd45f1f42c4d3197b5268e04a86043819ef67544ede6de72" gracePeriod=30
Nov 03 21:13:51.303205 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.303176    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 21:13:51.315303 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.315278    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-workload-certs\") pod \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") "
Nov 03 21:13:51.315411 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.315321    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-envoy\") pod \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") "
Nov 03 21:13:51.315411 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.315345    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68cccb32-d2a6-48f7-98f6-f0f60aab5496-kube-api-access-p859t\" (UniqueName: \"kubernetes.io/projected/68cccb32-d2a6-48f7-98f6-f0f60aab5496-kube-api-access-p859t\") pod \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") "
Nov 03 21:13:51.315411 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.315384    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-data\") pod \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") "
Nov 03 21:13:51.315411 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.315404    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-workload-socket\") pod \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") "
Nov 03 21:13:51.315649 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.315465    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istiod-ca-cert\") pod \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") "
Nov 03 21:13:51.315649 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.315498    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-token\" (UniqueName: \"kubernetes.io/projected/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-token\") pod \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") "
Nov 03 21:13:51.315649 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.315507    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-workload-certs" pod "68cccb32-d2a6-48f7-98f6-f0f60aab5496" (UID: "68cccb32-d2a6-48f7-98f6-f0f60aab5496"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:51.315649 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.315551    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-podinfo\") pod \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") "
Nov 03 21:13:51.315649 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.315591    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-credential-socket\") pod \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\" (UID: \"68cccb32-d2a6-48f7-98f6-f0f60aab5496\") "
Nov 03 21:13:51.315920 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.315769    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-data" pod "68cccb32-d2a6-48f7-98f6-f0f60aab5496" (UID: "68cccb32-d2a6-48f7-98f6-f0f60aab5496"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:51.315920 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.315846    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-workload-socket" pod "68cccb32-d2a6-48f7-98f6-f0f60aab5496" (UID: "68cccb32-d2a6-48f7-98f6-f0f60aab5496"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:51.316065 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.315940    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istiod-ca-cert" pod "68cccb32-d2a6-48f7-98f6-f0f60aab5496" (UID: "68cccb32-d2a6-48f7-98f6-f0f60aab5496"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:13:51.316135 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.316124    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:51.316352 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.316139    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:51.316352 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.316149    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:51.316352 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.316158    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:51.316352 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.316187    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-credential-socket" pod "68cccb32-d2a6-48f7-98f6-f0f60aab5496" (UID: "68cccb32-d2a6-48f7-98f6-f0f60aab5496"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:51.317612 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.317583    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-envoy" pod "68cccb32-d2a6-48f7-98f6-f0f60aab5496" (UID: "68cccb32-d2a6-48f7-98f6-f0f60aab5496"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:13:51.317743 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.317720    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68cccb32-d2a6-48f7-98f6-f0f60aab5496-kube-api-access-p859t" pod "68cccb32-d2a6-48f7-98f6-f0f60aab5496" (UID: "68cccb32-d2a6-48f7-98f6-f0f60aab5496"). InnerVolumeSpecName "kube-api-access-p859t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:51.317944 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.317921    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-podinfo" pod "68cccb32-d2a6-48f7-98f6-f0f60aab5496" (UID: "68cccb32-d2a6-48f7-98f6-f0f60aab5496"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:13:51.318079 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.318065    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-token" pod "68cccb32-d2a6-48f7-98f6-f0f60aab5496" (UID: "68cccb32-d2a6-48f7-98f6-f0f60aab5496"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:13:51.417208 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.417126    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:51.417208 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.417159    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p859t\" (UniqueName: \"kubernetes.io/projected/68cccb32-d2a6-48f7-98f6-f0f60aab5496-kube-api-access-p859t\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:51.417208 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.417174    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:51.417208 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.417186    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68cccb32-d2a6-48f7-98f6-f0f60aab5496-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:51.417208 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.417199    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68cccb32-d2a6-48f7-98f6-f0f60aab5496-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:13:51.435856 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.435821    4993 generic.go:356] "Generic (PLEG): container finished" podID="68cccb32-d2a6-48f7-98f6-f0f60aab5496" containerID="989be0d73dfc6cfdcd45f1f42c4d3197b5268e04a86043819ef67544ede6de72" exitCode=0
Nov 03 21:13:51.435997 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.435875    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"
Nov 03 21:13:51.435997 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.435901    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m" event={"ID":"68cccb32-d2a6-48f7-98f6-f0f60aab5496","Type":"ContainerDied","Data":"989be0d73dfc6cfdcd45f1f42c4d3197b5268e04a86043819ef67544ede6de72"}
Nov 03 21:13:51.435997 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.435954    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m" event={"ID":"68cccb32-d2a6-48f7-98f6-f0f60aab5496","Type":"ContainerDied","Data":"a9c00419054eb55be50209baf68a22beb440f940682b57398abf7be237c15d7b"}
Nov 03 21:13:51.435997 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.435977    4993 scope.go:122] "RemoveContainer" containerID="989be0d73dfc6cfdcd45f1f42c4d3197b5268e04a86043819ef67544ede6de72"
Nov 03 21:13:51.446087 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.446073    4993 scope.go:122] "RemoveContainer" containerID="989be0d73dfc6cfdcd45f1f42c4d3197b5268e04a86043819ef67544ede6de72"
Nov 03 21:13:51.446332 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:13:51.446313    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"989be0d73dfc6cfdcd45f1f42c4d3197b5268e04a86043819ef67544ede6de72\": container with ID starting with 989be0d73dfc6cfdcd45f1f42c4d3197b5268e04a86043819ef67544ede6de72 not found: ID does not exist" containerID="989be0d73dfc6cfdcd45f1f42c4d3197b5268e04a86043819ef67544ede6de72"
Nov 03 21:13:51.446401 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.446339    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"989be0d73dfc6cfdcd45f1f42c4d3197b5268e04a86043819ef67544ede6de72"} err="failed to get container status \"989be0d73dfc6cfdcd45f1f42c4d3197b5268e04a86043819ef67544ede6de72\": rpc error: code = NotFound desc = could not find container \"989be0d73dfc6cfdcd45f1f42c4d3197b5268e04a86043819ef67544ede6de72\": container with ID starting with 989be0d73dfc6cfdcd45f1f42c4d3197b5268e04a86043819ef67544ede6de72 not found: ID does not exist"
Nov 03 21:13:51.594039 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.594008    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"]
Nov 03 21:13:51.598894 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:51.598868    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d01cef-openshift-default-6d9778d4b5-txk6m"]
Nov 03 21:13:52.720998 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:52.720965    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"]
Nov 03 21:13:52.770758 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:52.770722    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"]
Nov 03 21:13:53.576723 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:53.576690    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68cccb32-d2a6-48f7-98f6-f0f60aab5496" path="/var/lib/kubelet/pods/68cccb32-d2a6-48f7-98f6-f0f60aab5496/volumes"
Nov 03 21:13:57.174012 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:57.173978    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"]
Nov 03 21:13:57.174476 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:57.174190    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5" podUID="45229f7a-8c58-4b55-b532-ff4aaf5032dd" containerName="istio-proxy" containerID="cri-o://0f95201ab6be0289048420dbacee187c3f116079901bec5b67eae5daf71ce481" gracePeriod=30
Nov 03 21:13:59.522136 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:59.522100    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"]
Nov 03 21:13:59.576002 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:13:59.575968    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"]
Nov 03 21:14:01.669584 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:14:01.669549    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:14:02.012983 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.012944    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:14:02.013197 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.013013    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5" podUID="45229f7a-8c58-4b55-b532-ff4aaf5032dd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:14:02.442794 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.442769    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:14:02.498952 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.498921    4993 generic.go:356] "Generic (PLEG): container finished" podID="45229f7a-8c58-4b55-b532-ff4aaf5032dd" containerID="0f95201ab6be0289048420dbacee187c3f116079901bec5b67eae5daf71ce481" exitCode=0
Nov 03 21:14:02.499119 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.498987    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"
Nov 03 21:14:02.499119 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.499004    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5" event={"ID":"45229f7a-8c58-4b55-b532-ff4aaf5032dd","Type":"ContainerDied","Data":"0f95201ab6be0289048420dbacee187c3f116079901bec5b67eae5daf71ce481"}
Nov 03 21:14:02.499119 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.499059    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5" event={"ID":"45229f7a-8c58-4b55-b532-ff4aaf5032dd","Type":"ContainerDied","Data":"f26dda8b56692638bfd05e53309ec4ac47a319b6fae097876fcccd0656c55eaf"}
Nov 03 21:14:02.499119 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.499082    4993 scope.go:122] "RemoveContainer" containerID="0f95201ab6be0289048420dbacee187c3f116079901bec5b67eae5daf71ce481"
Nov 03 21:14:02.509580 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.509558    4993 scope.go:122] "RemoveContainer" containerID="0f95201ab6be0289048420dbacee187c3f116079901bec5b67eae5daf71ce481"
Nov 03 21:14:02.509951 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:14:02.509925    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f95201ab6be0289048420dbacee187c3f116079901bec5b67eae5daf71ce481\": container with ID starting with 0f95201ab6be0289048420dbacee187c3f116079901bec5b67eae5daf71ce481 not found: ID does not exist" containerID="0f95201ab6be0289048420dbacee187c3f116079901bec5b67eae5daf71ce481"
Nov 03 21:14:02.510064 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.509966    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f95201ab6be0289048420dbacee187c3f116079901bec5b67eae5daf71ce481"} err="failed to get container status \"0f95201ab6be0289048420dbacee187c3f116079901bec5b67eae5daf71ce481\": rpc error: code = NotFound desc = could not find container \"0f95201ab6be0289048420dbacee187c3f116079901bec5b67eae5daf71ce481\": container with ID starting with 0f95201ab6be0289048420dbacee187c3f116079901bec5b67eae5daf71ce481 not found: ID does not exist"
Nov 03 21:14:02.555176 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.555113    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45229f7a-8c58-4b55-b532-ff4aaf5032dd-kube-api-access-n9s5x\" (UniqueName: \"kubernetes.io/projected/45229f7a-8c58-4b55-b532-ff4aaf5032dd-kube-api-access-n9s5x\") pod \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") "
Nov 03 21:14:02.555290 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.555178    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-credential-socket\") pod \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") "
Nov 03 21:14:02.555290 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.555209    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-envoy\") pod \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") "
Nov 03 21:14:02.555290 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.555260    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-podinfo\") pod \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") "
Nov 03 21:14:02.555427 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.555326    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-token\" (UniqueName: \"kubernetes.io/projected/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-token\") pod \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") "
Nov 03 21:14:02.555480 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.555435    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istiod-ca-cert\") pod \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") "
Nov 03 21:14:02.555538 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.555473    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-credential-socket" pod "45229f7a-8c58-4b55-b532-ff4aaf5032dd" (UID: "45229f7a-8c58-4b55-b532-ff4aaf5032dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:02.555608 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.555534    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-workload-certs\") pod \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") "
Nov 03 21:14:02.555608 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.555600    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-workload-socket\") pod \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") "
Nov 03 21:14:02.555735 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.555724    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-data\") pod \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\" (UID: \"45229f7a-8c58-4b55-b532-ff4aaf5032dd\") "
Nov 03 21:14:02.555821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.555796    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-workload-certs" pod "45229f7a-8c58-4b55-b532-ff4aaf5032dd" (UID: "45229f7a-8c58-4b55-b532-ff4aaf5032dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:02.555883 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.555827    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istiod-ca-cert" pod "45229f7a-8c58-4b55-b532-ff4aaf5032dd" (UID: "45229f7a-8c58-4b55-b532-ff4aaf5032dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:14:02.556223 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.556196    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-workload-socket" pod "45229f7a-8c58-4b55-b532-ff4aaf5032dd" (UID: "45229f7a-8c58-4b55-b532-ff4aaf5032dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:02.556318 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.556222    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:02.556383 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.556344    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:02.556383 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.556360    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:02.556497 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.556407    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-data" pod "45229f7a-8c58-4b55-b532-ff4aaf5032dd" (UID: "45229f7a-8c58-4b55-b532-ff4aaf5032dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:02.557549 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.557530    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45229f7a-8c58-4b55-b532-ff4aaf5032dd-kube-api-access-n9s5x" pod "45229f7a-8c58-4b55-b532-ff4aaf5032dd" (UID: "45229f7a-8c58-4b55-b532-ff4aaf5032dd"). InnerVolumeSpecName "kube-api-access-n9s5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:02.557671 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.557662    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-podinfo" pod "45229f7a-8c58-4b55-b532-ff4aaf5032dd" (UID: "45229f7a-8c58-4b55-b532-ff4aaf5032dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:14:02.557797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.557776    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-envoy" pod "45229f7a-8c58-4b55-b532-ff4aaf5032dd" (UID: "45229f7a-8c58-4b55-b532-ff4aaf5032dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:02.557852 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.557800    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-token" pod "45229f7a-8c58-4b55-b532-ff4aaf5032dd" (UID: "45229f7a-8c58-4b55-b532-ff4aaf5032dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:02.658009 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.657975    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:02.658009 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.658004    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:02.658009 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.658015    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:02.658242 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.658023    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:02.658242 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.658032    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9s5x\" (UniqueName: \"kubernetes.io/projected/45229f7a-8c58-4b55-b532-ff4aaf5032dd-kube-api-access-n9s5x\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:02.658242 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.658041    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45229f7a-8c58-4b55-b532-ff4aaf5032dd-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:02.924873 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.924838    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"]
Nov 03 21:14:02.932127 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:02.932103    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6n8q5"]
Nov 03 21:14:03.580593 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:03.580551    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="45229f7a-8c58-4b55-b532-ff4aaf5032dd" path="/var/lib/kubelet/pods/45229f7a-8c58-4b55-b532-ff4aaf5032dd/volumes"
Nov 03 21:14:07.742467 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:07.742424    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" containerID="cri-o://45197fcbe777ca1d03a201d639ff5641c1dc0bf0c0bf88cbdd095b1d2d729ffd" gracePeriod=45
Nov 03 21:14:07.791130 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:07.791083    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main" containerID="cri-o://0ef61ee1499c0063b5a8521797cc65fff45e42f1d85e633f1130cc7a0edf392a" gracePeriod=45
Nov 03 21:14:07.795929 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:07.795889    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": dial tcp 10.133.0.51:8000: connect: connection refused"
Nov 03 21:14:08.081690 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.078426    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 21:14:08.123666 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.123606    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 21:14:08.220455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.219145    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-tmp-dir\") pod \"81f1a530-ef53-4782-aae4-4a8666859325\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") "
Nov 03 21:14:08.220455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.219332    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/81f1a530-ef53-4782-aae4-4a8666859325-tls-certs\" (UniqueName: \"kubernetes.io/secret/81f1a530-ef53-4782-aae4-4a8666859325-tls-certs\") pod \"81f1a530-ef53-4782-aae4-4a8666859325\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") "
Nov 03 21:14:08.220455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.219372    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2204dd40-a737-47fc-a685-420d5b819a29-tls-certs\" (UniqueName: \"kubernetes.io/secret/2204dd40-a737-47fc-a685-420d5b819a29-tls-certs\") pod \"2204dd40-a737-47fc-a685-420d5b819a29\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") "
Nov 03 21:14:08.220455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.219389    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-tmp-dir" pod "81f1a530-ef53-4782-aae4-4a8666859325" (UID: "81f1a530-ef53-4782-aae4-4a8666859325"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:08.220455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.219411    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-home\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-home\") pod \"81f1a530-ef53-4782-aae4-4a8666859325\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") "
Nov 03 21:14:08.220455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.219890    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-model-cache\") pod \"81f1a530-ef53-4782-aae4-4a8666859325\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") "
Nov 03 21:14:08.220455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.219984    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81f1a530-ef53-4782-aae4-4a8666859325-kube-api-access-kxk6w\" (UniqueName: \"kubernetes.io/projected/81f1a530-ef53-4782-aae4-4a8666859325-kube-api-access-kxk6w\") pod \"81f1a530-ef53-4782-aae4-4a8666859325\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") "
Nov 03 21:14:08.220455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.220029    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2204dd40-a737-47fc-a685-420d5b819a29-kube-api-access-dx6dj\" (UniqueName: \"kubernetes.io/projected/2204dd40-a737-47fc-a685-420d5b819a29-kube-api-access-dx6dj\") pod \"2204dd40-a737-47fc-a685-420d5b819a29\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") "
Nov 03 21:14:08.220455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.220111    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-home\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-home\") pod \"2204dd40-a737-47fc-a685-420d5b819a29\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") "
Nov 03 21:14:08.220455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.220139    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-dshm\") pod \"2204dd40-a737-47fc-a685-420d5b819a29\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") "
Nov 03 21:14:08.220455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.220173    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-tmp-dir\") pod \"2204dd40-a737-47fc-a685-420d5b819a29\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") "
Nov 03 21:14:08.220455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.220171    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-model-cache" pod "81f1a530-ef53-4782-aae4-4a8666859325" (UID: "81f1a530-ef53-4782-aae4-4a8666859325"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:08.220455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.220203    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-dshm\") pod \"81f1a530-ef53-4782-aae4-4a8666859325\" (UID: \"81f1a530-ef53-4782-aae4-4a8666859325\") "
Nov 03 21:14:08.220455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.220262    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-model-cache\") pod \"2204dd40-a737-47fc-a685-420d5b819a29\" (UID: \"2204dd40-a737-47fc-a685-420d5b819a29\") "
Nov 03 21:14:08.220455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.220365    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-home" pod "2204dd40-a737-47fc-a685-420d5b819a29" (UID: "2204dd40-a737-47fc-a685-420d5b819a29"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:08.224998 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.220497    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-tmp-dir" pod "2204dd40-a737-47fc-a685-420d5b819a29" (UID: "2204dd40-a737-47fc-a685-420d5b819a29"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:08.224998 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.220692    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-model-cache" pod "2204dd40-a737-47fc-a685-420d5b819a29" (UID: "2204dd40-a737-47fc-a685-420d5b819a29"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:08.224998 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.220804    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:08.224998 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.220824    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:08.224998 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.220838    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:08.224998 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.220853    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:08.224998 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.220868    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:08.224998 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.223772    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-home" pod "81f1a530-ef53-4782-aae4-4a8666859325" (UID: "81f1a530-ef53-4782-aae4-4a8666859325"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:08.227334 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.227305    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-dshm" pod "2204dd40-a737-47fc-a685-420d5b819a29" (UID: "2204dd40-a737-47fc-a685-420d5b819a29"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:08.227556 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.227516    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2204dd40-a737-47fc-a685-420d5b819a29-tls-certs" pod "2204dd40-a737-47fc-a685-420d5b819a29" (UID: "2204dd40-a737-47fc-a685-420d5b819a29"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:08.227732 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.227711    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81f1a530-ef53-4782-aae4-4a8666859325-kube-api-access-kxk6w" pod "81f1a530-ef53-4782-aae4-4a8666859325" (UID: "81f1a530-ef53-4782-aae4-4a8666859325"). InnerVolumeSpecName "kube-api-access-kxk6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:08.227820 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.227792    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2204dd40-a737-47fc-a685-420d5b819a29-kube-api-access-dx6dj" pod "2204dd40-a737-47fc-a685-420d5b819a29" (UID: "2204dd40-a737-47fc-a685-420d5b819a29"). InnerVolumeSpecName "kube-api-access-dx6dj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:08.227820 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.227813    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81f1a530-ef53-4782-aae4-4a8666859325-tls-certs" pod "81f1a530-ef53-4782-aae4-4a8666859325" (UID: "81f1a530-ef53-4782-aae4-4a8666859325"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:08.228993 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.228971    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-dshm" pod "81f1a530-ef53-4782-aae4-4a8666859325" (UID: "81f1a530-ef53-4782-aae4-4a8666859325"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:08.322586 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.322504    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/81f1a530-ef53-4782-aae4-4a8666859325-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:08.322586 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.322532    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2204dd40-a737-47fc-a685-420d5b819a29-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:08.322586 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.322544    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:08.322586 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.322559    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxk6w\" (UniqueName: \"kubernetes.io/projected/81f1a530-ef53-4782-aae4-4a8666859325-kube-api-access-kxk6w\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:08.322586 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.322568    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx6dj\" (UniqueName: \"kubernetes.io/projected/2204dd40-a737-47fc-a685-420d5b819a29-kube-api-access-dx6dj\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:08.322586 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.322576    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2204dd40-a737-47fc-a685-420d5b819a29-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:08.322586 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.322584    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/81f1a530-ef53-4782-aae4-4a8666859325-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:08.538507 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.538470    4993 generic.go:356] "Generic (PLEG): container finished" podID="2204dd40-a737-47fc-a685-420d5b819a29" containerID="0ef61ee1499c0063b5a8521797cc65fff45e42f1d85e633f1130cc7a0edf392a" exitCode=0
Nov 03 21:14:08.538770 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.538535    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"
Nov 03 21:14:08.538770 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.538577    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" event={"ID":"2204dd40-a737-47fc-a685-420d5b819a29","Type":"ContainerDied","Data":"0ef61ee1499c0063b5a8521797cc65fff45e42f1d85e633f1130cc7a0edf392a"}
Nov 03 21:14:08.538770 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.538651    4993 scope.go:122] "RemoveContainer" containerID="0ef61ee1499c0063b5a8521797cc65fff45e42f1d85e633f1130cc7a0edf392a"
Nov 03 21:14:08.538770 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.538655    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv" event={"ID":"2204dd40-a737-47fc-a685-420d5b819a29","Type":"ContainerDied","Data":"96e1f18265ad9a47fc3913953bda5797b19ef016aa9a8404ae62c8c05ddd419c"}
Nov 03 21:14:08.541480 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.541450    4993 generic.go:356] "Generic (PLEG): container finished" podID="81f1a530-ef53-4782-aae4-4a8666859325" containerID="45197fcbe777ca1d03a201d639ff5641c1dc0bf0c0bf88cbdd095b1d2d729ffd" exitCode=0
Nov 03 21:14:08.541656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.541570    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" event={"ID":"81f1a530-ef53-4782-aae4-4a8666859325","Type":"ContainerDied","Data":"45197fcbe777ca1d03a201d639ff5641c1dc0bf0c0bf88cbdd095b1d2d729ffd"}
Nov 03 21:14:08.541656 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.541601    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454" event={"ID":"81f1a530-ef53-4782-aae4-4a8666859325","Type":"ContainerDied","Data":"47409e97bb03b13632c11a30aa520e7a33c284fb9a14b26738ba1c98a0948ab0"}
Nov 03 21:14:08.541909 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.541676    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"
Nov 03 21:14:08.554618 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.554596    4993 scope.go:122] "RemoveContainer" containerID="0ef61ee1499c0063b5a8521797cc65fff45e42f1d85e633f1130cc7a0edf392a"
Nov 03 21:14:08.554929 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:14:08.554911    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ef61ee1499c0063b5a8521797cc65fff45e42f1d85e633f1130cc7a0edf392a\": container with ID starting with 0ef61ee1499c0063b5a8521797cc65fff45e42f1d85e633f1130cc7a0edf392a not found: ID does not exist" containerID="0ef61ee1499c0063b5a8521797cc65fff45e42f1d85e633f1130cc7a0edf392a"
Nov 03 21:14:08.554985 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.554939    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ef61ee1499c0063b5a8521797cc65fff45e42f1d85e633f1130cc7a0edf392a"} err="failed to get container status \"0ef61ee1499c0063b5a8521797cc65fff45e42f1d85e633f1130cc7a0edf392a\": rpc error: code = NotFound desc = could not find container \"0ef61ee1499c0063b5a8521797cc65fff45e42f1d85e633f1130cc7a0edf392a\": container with ID starting with 0ef61ee1499c0063b5a8521797cc65fff45e42f1d85e633f1130cc7a0edf392a not found: ID does not exist"
Nov 03 21:14:08.554985 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.554959    4993 scope.go:122] "RemoveContainer" containerID="45197fcbe777ca1d03a201d639ff5641c1dc0bf0c0bf88cbdd095b1d2d729ffd"
Nov 03 21:14:08.563159 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.563139    4993 scope.go:122] "RemoveContainer" containerID="45197fcbe777ca1d03a201d639ff5641c1dc0bf0c0bf88cbdd095b1d2d729ffd"
Nov 03 21:14:08.563446 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:14:08.563428    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45197fcbe777ca1d03a201d639ff5641c1dc0bf0c0bf88cbdd095b1d2d729ffd\": container with ID starting with 45197fcbe777ca1d03a201d639ff5641c1dc0bf0c0bf88cbdd095b1d2d729ffd not found: ID does not exist" containerID="45197fcbe777ca1d03a201d639ff5641c1dc0bf0c0bf88cbdd095b1d2d729ffd"
Nov 03 21:14:08.563503 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.563454    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45197fcbe777ca1d03a201d639ff5641c1dc0bf0c0bf88cbdd095b1d2d729ffd"} err="failed to get container status \"45197fcbe777ca1d03a201d639ff5641c1dc0bf0c0bf88cbdd095b1d2d729ffd\": rpc error: code = NotFound desc = could not find container \"45197fcbe777ca1d03a201d639ff5641c1dc0bf0c0bf88cbdd095b1d2d729ffd\": container with ID starting with 45197fcbe777ca1d03a201d639ff5641c1dc0bf0c0bf88cbdd095b1d2d729ffd not found: ID does not exist"
Nov 03 21:14:08.703424 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.703342    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"]
Nov 03 21:14:08.714119 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.713703    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-e22abf/simulator-w5-e22abf-kserve-67d48d4c4-jrpgv"]
Nov 03 21:14:08.820735 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.820701    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"]
Nov 03 21:14:08.824773 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:08.824748    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-e22abf/distinct-w5-e22abf-kserve-dd558dcc8-2h454"]
Nov 03 21:14:09.576263 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:09.576226    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2204dd40-a737-47fc-a685-420d5b819a29" path="/var/lib/kubelet/pods/2204dd40-a737-47fc-a685-420d5b819a29/volumes"
Nov 03 21:14:09.576666 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:09.576648    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81f1a530-ef53-4782-aae4-4a8666859325" path="/var/lib/kubelet/pods/81f1a530-ef53-4782-aae4-4a8666859325/volumes"
Nov 03 21:14:11.102083 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:11.102050    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"]
Nov 03 21:14:11.121711 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:11.121682    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"]
Nov 03 21:14:14.542950 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:14.542866    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" containerID="cri-o://d392c6f8a4fe5ffa8d4cf7acb16aba418f3bd7cf826961181f8ead5408fd5915" gracePeriod=45
Nov 03 21:14:14.596220 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:14.596185    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main" containerID="cri-o://c0a7bd9a7fccc6ef9659fef3a769eec695eb197c4acf11f42c589c323cfc1fa5" gracePeriod=45
Nov 03 21:14:14.870953 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:14.870924    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 21:14:14.888148 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:14.888125    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 21:14:15.003691 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.003657    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-model-cache\") pod \"b64da81f-0a97-4cdd-9afd-1d58a306c717\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") "
Nov 03 21:14:15.003845 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.003764    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-model-cache\") pod \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") "
Nov 03 21:14:15.003845 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.003821    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-tmp-dir\") pod \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") "
Nov 03 21:14:15.003944 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.003854    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-home\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-home\") pod \"b64da81f-0a97-4cdd-9afd-1d58a306c717\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") "
Nov 03 21:14:15.003944 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.003889    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-model-cache" pod "b64da81f-0a97-4cdd-9afd-1d58a306c717" (UID: "b64da81f-0a97-4cdd-9afd-1d58a306c717"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:15.003944 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.003915    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-tmp-dir\") pod \"b64da81f-0a97-4cdd-9afd-1d58a306c717\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") "
Nov 03 21:14:15.004105 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.003965    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-dshm\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-dshm\") pod \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") "
Nov 03 21:14:15.004105 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004026    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-home\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-home\") pod \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") "
Nov 03 21:14:15.004105 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004058    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/809df5f7-98f7-4dbf-8548-b0f7dade9c40-tls-certs\" (UniqueName: \"kubernetes.io/secret/809df5f7-98f7-4dbf-8548-b0f7dade9c40-tls-certs\") pod \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") "
Nov 03 21:14:15.004105 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004076    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-model-cache" pod "809df5f7-98f7-4dbf-8548-b0f7dade9c40" (UID: "809df5f7-98f7-4dbf-8548-b0f7dade9c40"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:15.004306 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004129    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-home" pod "b64da81f-0a97-4cdd-9afd-1d58a306c717" (UID: "b64da81f-0a97-4cdd-9afd-1d58a306c717"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:15.004306 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004160    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-tmp-dir" pod "809df5f7-98f7-4dbf-8548-b0f7dade9c40" (UID: "809df5f7-98f7-4dbf-8548-b0f7dade9c40"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:15.004306 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004176    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b64da81f-0a97-4cdd-9afd-1d58a306c717-kube-api-access-j5mdc\" (UniqueName: \"kubernetes.io/projected/b64da81f-0a97-4cdd-9afd-1d58a306c717-kube-api-access-j5mdc\") pod \"b64da81f-0a97-4cdd-9afd-1d58a306c717\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") "
Nov 03 21:14:15.004306 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004190    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-tmp-dir" pod "b64da81f-0a97-4cdd-9afd-1d58a306c717" (UID: "b64da81f-0a97-4cdd-9afd-1d58a306c717"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:15.004306 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004205    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-dshm\") pod \"b64da81f-0a97-4cdd-9afd-1d58a306c717\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") "
Nov 03 21:14:15.004306 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004232    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/809df5f7-98f7-4dbf-8548-b0f7dade9c40-kube-api-access-wgqbb\" (UniqueName: \"kubernetes.io/projected/809df5f7-98f7-4dbf-8548-b0f7dade9c40-kube-api-access-wgqbb\") pod \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\" (UID: \"809df5f7-98f7-4dbf-8548-b0f7dade9c40\") "
Nov 03 21:14:15.004306 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004275    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b64da81f-0a97-4cdd-9afd-1d58a306c717-tls-certs\" (UniqueName: \"kubernetes.io/secret/b64da81f-0a97-4cdd-9afd-1d58a306c717-tls-certs\") pod \"b64da81f-0a97-4cdd-9afd-1d58a306c717\" (UID: \"b64da81f-0a97-4cdd-9afd-1d58a306c717\") "
Nov 03 21:14:15.004681 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004352    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-home" pod "809df5f7-98f7-4dbf-8548-b0f7dade9c40" (UID: "809df5f7-98f7-4dbf-8548-b0f7dade9c40"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:15.004771 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004694    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.004771 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004715    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.004771 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004728    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.004771 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004741    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.004771 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004754    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.004771 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.004765    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.006989 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.006959    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-dshm" pod "809df5f7-98f7-4dbf-8548-b0f7dade9c40" (UID: "809df5f7-98f7-4dbf-8548-b0f7dade9c40"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:15.007117 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.007057    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/809df5f7-98f7-4dbf-8548-b0f7dade9c40-kube-api-access-wgqbb" pod "809df5f7-98f7-4dbf-8548-b0f7dade9c40" (UID: "809df5f7-98f7-4dbf-8548-b0f7dade9c40"). InnerVolumeSpecName "kube-api-access-wgqbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:15.007192 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.007163    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/809df5f7-98f7-4dbf-8548-b0f7dade9c40-tls-certs" pod "809df5f7-98f7-4dbf-8548-b0f7dade9c40" (UID: "809df5f7-98f7-4dbf-8548-b0f7dade9c40"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:15.007271 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.007253    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b64da81f-0a97-4cdd-9afd-1d58a306c717-tls-certs" pod "b64da81f-0a97-4cdd-9afd-1d58a306c717" (UID: "b64da81f-0a97-4cdd-9afd-1d58a306c717"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:15.007550 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.007519    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b64da81f-0a97-4cdd-9afd-1d58a306c717-kube-api-access-j5mdc" pod "b64da81f-0a97-4cdd-9afd-1d58a306c717" (UID: "b64da81f-0a97-4cdd-9afd-1d58a306c717"). InnerVolumeSpecName "kube-api-access-j5mdc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:15.007604 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.007584    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-dshm" pod "b64da81f-0a97-4cdd-9afd-1d58a306c717" (UID: "b64da81f-0a97-4cdd-9afd-1d58a306c717"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:15.105987 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.105895    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b64da81f-0a97-4cdd-9afd-1d58a306c717-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.105987 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.105931    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgqbb\" (UniqueName: \"kubernetes.io/projected/809df5f7-98f7-4dbf-8548-b0f7dade9c40-kube-api-access-wgqbb\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.105987 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.105946    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b64da81f-0a97-4cdd-9afd-1d58a306c717-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.105987 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.105960    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/809df5f7-98f7-4dbf-8548-b0f7dade9c40-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.105987 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.105972    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/809df5f7-98f7-4dbf-8548-b0f7dade9c40-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.105987 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.105986    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5mdc\" (UniqueName: \"kubernetes.io/projected/b64da81f-0a97-4cdd-9afd-1d58a306c717-kube-api-access-j5mdc\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:15.580041 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.580005    4993 generic.go:356] "Generic (PLEG): container finished" podID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerID="c0a7bd9a7fccc6ef9659fef3a769eec695eb197c4acf11f42c589c323cfc1fa5" exitCode=0
Nov 03 21:14:15.580466 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.580089    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"
Nov 03 21:14:15.580466 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.580106    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" event={"ID":"b64da81f-0a97-4cdd-9afd-1d58a306c717","Type":"ContainerDied","Data":"c0a7bd9a7fccc6ef9659fef3a769eec695eb197c4acf11f42c589c323cfc1fa5"}
Nov 03 21:14:15.580466 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.580166    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd" event={"ID":"b64da81f-0a97-4cdd-9afd-1d58a306c717","Type":"ContainerDied","Data":"067c38117a8148cae649bdba60fbecd92ef6375474bfe486be2a2ae7a828affa"}
Nov 03 21:14:15.580466 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.580189    4993 scope.go:122] "RemoveContainer" containerID="c0a7bd9a7fccc6ef9659fef3a769eec695eb197c4acf11f42c589c323cfc1fa5"
Nov 03 21:14:15.582121 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.582098    4993 generic.go:356] "Generic (PLEG): container finished" podID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerID="d392c6f8a4fe5ffa8d4cf7acb16aba418f3bd7cf826961181f8ead5408fd5915" exitCode=0
Nov 03 21:14:15.582230 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.582140    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" event={"ID":"809df5f7-98f7-4dbf-8548-b0f7dade9c40","Type":"ContainerDied","Data":"d392c6f8a4fe5ffa8d4cf7acb16aba418f3bd7cf826961181f8ead5408fd5915"}
Nov 03 21:14:15.582230 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.582166    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" event={"ID":"809df5f7-98f7-4dbf-8548-b0f7dade9c40","Type":"ContainerDied","Data":"c958689a22672fe577223e4f622d5fcf0c5615236036676b8e154b4127d221c9"}
Nov 03 21:14:15.582230 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.582166    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"
Nov 03 21:14:15.593015 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.592994    4993 scope.go:122] "RemoveContainer" containerID="c0a7bd9a7fccc6ef9659fef3a769eec695eb197c4acf11f42c589c323cfc1fa5"
Nov 03 21:14:15.593551 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:14:15.593506    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0a7bd9a7fccc6ef9659fef3a769eec695eb197c4acf11f42c589c323cfc1fa5\": container with ID starting with c0a7bd9a7fccc6ef9659fef3a769eec695eb197c4acf11f42c589c323cfc1fa5 not found: ID does not exist" containerID="c0a7bd9a7fccc6ef9659fef3a769eec695eb197c4acf11f42c589c323cfc1fa5"
Nov 03 21:14:15.593680 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.593548    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0a7bd9a7fccc6ef9659fef3a769eec695eb197c4acf11f42c589c323cfc1fa5"} err="failed to get container status \"c0a7bd9a7fccc6ef9659fef3a769eec695eb197c4acf11f42c589c323cfc1fa5\": rpc error: code = NotFound desc = could not find container \"c0a7bd9a7fccc6ef9659fef3a769eec695eb197c4acf11f42c589c323cfc1fa5\": container with ID starting with c0a7bd9a7fccc6ef9659fef3a769eec695eb197c4acf11f42c589c323cfc1fa5 not found: ID does not exist"
Nov 03 21:14:15.593680 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.593571    4993 scope.go:122] "RemoveContainer" containerID="d392c6f8a4fe5ffa8d4cf7acb16aba418f3bd7cf826961181f8ead5408fd5915"
Nov 03 21:14:15.602404 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.602378    4993 scope.go:122] "RemoveContainer" containerID="d392c6f8a4fe5ffa8d4cf7acb16aba418f3bd7cf826961181f8ead5408fd5915"
Nov 03 21:14:15.602689 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:14:15.602667    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d392c6f8a4fe5ffa8d4cf7acb16aba418f3bd7cf826961181f8ead5408fd5915\": container with ID starting with d392c6f8a4fe5ffa8d4cf7acb16aba418f3bd7cf826961181f8ead5408fd5915 not found: ID does not exist" containerID="d392c6f8a4fe5ffa8d4cf7acb16aba418f3bd7cf826961181f8ead5408fd5915"
Nov 03 21:14:15.602772 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.602694    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d392c6f8a4fe5ffa8d4cf7acb16aba418f3bd7cf826961181f8ead5408fd5915"} err="failed to get container status \"d392c6f8a4fe5ffa8d4cf7acb16aba418f3bd7cf826961181f8ead5408fd5915\": rpc error: code = NotFound desc = could not find container \"d392c6f8a4fe5ffa8d4cf7acb16aba418f3bd7cf826961181f8ead5408fd5915\": container with ID starting with d392c6f8a4fe5ffa8d4cf7acb16aba418f3bd7cf826961181f8ead5408fd5915 not found: ID does not exist"
Nov 03 21:14:15.745426 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.745386    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"]
Nov 03 21:14:15.795682 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.795619    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk"]
Nov 03 21:14:15.804328 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.804290    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-12a423/premium-w6-12a423-kserve-869cd6648c-hj2hk" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main" probeResult="failure" output="Get \"https://10.133.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:14:15.903761 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.903685    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"]
Nov 03 21:14:15.910206 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:15.910175    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-12a423/simulator-w6-12a423-kserve-7b547b585d-2t5kd"]
Nov 03 21:14:17.576376 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:17.576343    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" path="/var/lib/kubelet/pods/809df5f7-98f7-4dbf-8548-b0f7dade9c40/volumes"
Nov 03 21:14:17.576775 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:17.576761    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" path="/var/lib/kubelet/pods/b64da81f-0a97-4cdd-9afd-1d58a306c717/volumes"
Nov 03 21:14:26.143207 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.143144    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main" containerID="cri-o://037cd997fbc22fe0fdf8443cecc5a602c4dd3b09095e1abc626464ebcdc68801" gracePeriod=45
Nov 03 21:14:26.397291 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.397237    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 21:14:26.419317 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.419296    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-model-cache\") pod \"4ff97541-97f9-4ff6-b218-c717565ef6a4\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") "
Nov 03 21:14:26.419402 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.419357    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-home\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-home\") pod \"4ff97541-97f9-4ff6-b218-c717565ef6a4\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") "
Nov 03 21:14:26.419402 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.419397    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-tmp-dir\") pod \"4ff97541-97f9-4ff6-b218-c717565ef6a4\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") "
Nov 03 21:14:26.419524 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.419427    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ff97541-97f9-4ff6-b218-c717565ef6a4-kube-api-access-8ktbs\" (UniqueName: \"kubernetes.io/projected/4ff97541-97f9-4ff6-b218-c717565ef6a4-kube-api-access-8ktbs\") pod \"4ff97541-97f9-4ff6-b218-c717565ef6a4\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") "
Nov 03 21:14:26.419524 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.419516    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-dshm\") pod \"4ff97541-97f9-4ff6-b218-c717565ef6a4\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") "
Nov 03 21:14:26.419657 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.419554    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4ff97541-97f9-4ff6-b218-c717565ef6a4-tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff97541-97f9-4ff6-b218-c717565ef6a4-tls-certs\") pod \"4ff97541-97f9-4ff6-b218-c717565ef6a4\" (UID: \"4ff97541-97f9-4ff6-b218-c717565ef6a4\") "
Nov 03 21:14:26.419657 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.419606    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-model-cache" pod "4ff97541-97f9-4ff6-b218-c717565ef6a4" (UID: "4ff97541-97f9-4ff6-b218-c717565ef6a4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.419754 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.419684    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-home" pod "4ff97541-97f9-4ff6-b218-c717565ef6a4" (UID: "4ff97541-97f9-4ff6-b218-c717565ef6a4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.419993 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.419971    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.420090 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.419999    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.420090 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.420023    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-tmp-dir" pod "4ff97541-97f9-4ff6-b218-c717565ef6a4" (UID: "4ff97541-97f9-4ff6-b218-c717565ef6a4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.422088 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.422061    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ff97541-97f9-4ff6-b218-c717565ef6a4-kube-api-access-8ktbs" pod "4ff97541-97f9-4ff6-b218-c717565ef6a4" (UID: "4ff97541-97f9-4ff6-b218-c717565ef6a4"). InnerVolumeSpecName "kube-api-access-8ktbs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:26.422088 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.422077    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-dshm" pod "4ff97541-97f9-4ff6-b218-c717565ef6a4" (UID: "4ff97541-97f9-4ff6-b218-c717565ef6a4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.422225 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.422086    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ff97541-97f9-4ff6-b218-c717565ef6a4-tls-certs" pod "4ff97541-97f9-4ff6-b218-c717565ef6a4" (UID: "4ff97541-97f9-4ff6-b218-c717565ef6a4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:26.521256 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.521226    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.521256 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.521259    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ktbs\" (UniqueName: \"kubernetes.io/projected/4ff97541-97f9-4ff6-b218-c717565ef6a4-kube-api-access-8ktbs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.521460 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.521273    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff97541-97f9-4ff6-b218-c717565ef6a4-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.521460 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.521285    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff97541-97f9-4ff6-b218-c717565ef6a4-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.579711 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.579653    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" containerID="cri-o://aa03008a9ce10d11ac59d7299076fb98bdc5dea3722df8f90485ec1057531163" gracePeriod=45
Nov 03 21:14:26.656374 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.656295    4993 generic.go:356] "Generic (PLEG): container finished" podID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerID="037cd997fbc22fe0fdf8443cecc5a602c4dd3b09095e1abc626464ebcdc68801" exitCode=0
Nov 03 21:14:26.656374 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.656337    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"
Nov 03 21:14:26.656555 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.656378    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" event={"ID":"4ff97541-97f9-4ff6-b218-c717565ef6a4","Type":"ContainerDied","Data":"037cd997fbc22fe0fdf8443cecc5a602c4dd3b09095e1abc626464ebcdc68801"}
Nov 03 21:14:26.656555 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.656422    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n" event={"ID":"4ff97541-97f9-4ff6-b218-c717565ef6a4","Type":"ContainerDied","Data":"2af784417810a3be5bf4c4e9c8686a74e60f4960495cfc7cb4fb65596d5695b2"}
Nov 03 21:14:26.656555 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.656440    4993 scope.go:122] "RemoveContainer" containerID="037cd997fbc22fe0fdf8443cecc5a602c4dd3b09095e1abc626464ebcdc68801"
Nov 03 21:14:26.693141 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.693116    4993 scope.go:122] "RemoveContainer" containerID="037cd997fbc22fe0fdf8443cecc5a602c4dd3b09095e1abc626464ebcdc68801"
Nov 03 21:14:26.693433 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:14:26.693415    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"037cd997fbc22fe0fdf8443cecc5a602c4dd3b09095e1abc626464ebcdc68801\": container with ID starting with 037cd997fbc22fe0fdf8443cecc5a602c4dd3b09095e1abc626464ebcdc68801 not found: ID does not exist" containerID="037cd997fbc22fe0fdf8443cecc5a602c4dd3b09095e1abc626464ebcdc68801"
Nov 03 21:14:26.693487 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.693444    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"037cd997fbc22fe0fdf8443cecc5a602c4dd3b09095e1abc626464ebcdc68801"} err="failed to get container status \"037cd997fbc22fe0fdf8443cecc5a602c4dd3b09095e1abc626464ebcdc68801\": rpc error: code = NotFound desc = could not find container \"037cd997fbc22fe0fdf8443cecc5a602c4dd3b09095e1abc626464ebcdc68801\": container with ID starting with 037cd997fbc22fe0fdf8443cecc5a602c4dd3b09095e1abc626464ebcdc68801 not found: ID does not exist"
Nov 03 21:14:26.821093 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.821066    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"]
Nov 03 21:14:26.826683 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.826659    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:14:26.844439 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.844411    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-862138/premium-w2-862138-kserve-5dd7d4cdd-scr2n"]
Nov 03 21:14:26.926168 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.926094    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8fe2fef6-f3ea-49a2-aa27-47635359b017-tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe2fef6-f3ea-49a2-aa27-47635359b017-tls-certs\") pod \"8fe2fef6-f3ea-49a2-aa27-47635359b017\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") "
Nov 03 21:14:26.926168 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.926150    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-model-cache\") pod \"8fe2fef6-f3ea-49a2-aa27-47635359b017\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") "
Nov 03 21:14:26.926365 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.926185    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-tmp-dir\") pod \"8fe2fef6-f3ea-49a2-aa27-47635359b017\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") "
Nov 03 21:14:26.926365 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.926205    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-dshm\") pod \"8fe2fef6-f3ea-49a2-aa27-47635359b017\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") "
Nov 03 21:14:26.926365 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.926227    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-home\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-home\") pod \"8fe2fef6-f3ea-49a2-aa27-47635359b017\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") "
Nov 03 21:14:26.926365 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.926273    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8fe2fef6-f3ea-49a2-aa27-47635359b017-kube-api-access-49gxw\" (UniqueName: \"kubernetes.io/projected/8fe2fef6-f3ea-49a2-aa27-47635359b017-kube-api-access-49gxw\") pod \"8fe2fef6-f3ea-49a2-aa27-47635359b017\" (UID: \"8fe2fef6-f3ea-49a2-aa27-47635359b017\") "
Nov 03 21:14:26.926595 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.926471    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-model-cache" pod "8fe2fef6-f3ea-49a2-aa27-47635359b017" (UID: "8fe2fef6-f3ea-49a2-aa27-47635359b017"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.926595 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.926483    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-tmp-dir" pod "8fe2fef6-f3ea-49a2-aa27-47635359b017" (UID: "8fe2fef6-f3ea-49a2-aa27-47635359b017"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.926595 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.926538    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-home" pod "8fe2fef6-f3ea-49a2-aa27-47635359b017" (UID: "8fe2fef6-f3ea-49a2-aa27-47635359b017"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.927038 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.927016    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.927111 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.927045    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.927111 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.927060    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:26.928364 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.928344    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-dshm" pod "8fe2fef6-f3ea-49a2-aa27-47635359b017" (UID: "8fe2fef6-f3ea-49a2-aa27-47635359b017"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:26.928364 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.928354    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fe2fef6-f3ea-49a2-aa27-47635359b017-tls-certs" pod "8fe2fef6-f3ea-49a2-aa27-47635359b017" (UID: "8fe2fef6-f3ea-49a2-aa27-47635359b017"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:26.928468 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:26.928429    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fe2fef6-f3ea-49a2-aa27-47635359b017-kube-api-access-49gxw" pod "8fe2fef6-f3ea-49a2-aa27-47635359b017" (UID: "8fe2fef6-f3ea-49a2-aa27-47635359b017"). InnerVolumeSpecName "kube-api-access-49gxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:27.027596 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:27.027559    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fe2fef6-f3ea-49a2-aa27-47635359b017-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:27.027596 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:27.027589    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fe2fef6-f3ea-49a2-aa27-47635359b017-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:27.027596 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:27.027599    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49gxw\" (UniqueName: \"kubernetes.io/projected/8fe2fef6-f3ea-49a2-aa27-47635359b017-kube-api-access-49gxw\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:27.576020 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:27.575988    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" path="/var/lib/kubelet/pods/4ff97541-97f9-4ff6-b218-c717565ef6a4/volumes"
Nov 03 21:14:27.662906 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:27.662874    4993 generic.go:356] "Generic (PLEG): container finished" podID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerID="aa03008a9ce10d11ac59d7299076fb98bdc5dea3722df8f90485ec1057531163" exitCode=0
Nov 03 21:14:27.663047 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:27.662923    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"
Nov 03 21:14:27.663047 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:27.662921    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" event={"ID":"8fe2fef6-f3ea-49a2-aa27-47635359b017","Type":"ContainerDied","Data":"aa03008a9ce10d11ac59d7299076fb98bdc5dea3722df8f90485ec1057531163"}
Nov 03 21:14:27.663047 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:27.663033    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" event={"ID":"8fe2fef6-f3ea-49a2-aa27-47635359b017","Type":"ContainerDied","Data":"fba58e00fd0265cc97fb4c3f5d244bb1337a41eb5f91ba493ea35c03c27f14fa"}
Nov 03 21:14:27.663208 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:27.663050    4993 scope.go:122] "RemoveContainer" containerID="aa03008a9ce10d11ac59d7299076fb98bdc5dea3722df8f90485ec1057531163"
Nov 03 21:14:27.671663 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:27.671619    4993 scope.go:122] "RemoveContainer" containerID="aa03008a9ce10d11ac59d7299076fb98bdc5dea3722df8f90485ec1057531163"
Nov 03 21:14:27.671923 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:14:27.671895    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa03008a9ce10d11ac59d7299076fb98bdc5dea3722df8f90485ec1057531163\": container with ID starting with aa03008a9ce10d11ac59d7299076fb98bdc5dea3722df8f90485ec1057531163 not found: ID does not exist" containerID="aa03008a9ce10d11ac59d7299076fb98bdc5dea3722df8f90485ec1057531163"
Nov 03 21:14:27.671999 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:27.671924    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa03008a9ce10d11ac59d7299076fb98bdc5dea3722df8f90485ec1057531163"} err="failed to get container status \"aa03008a9ce10d11ac59d7299076fb98bdc5dea3722df8f90485ec1057531163\": rpc error: code = NotFound desc = could not find container \"aa03008a9ce10d11ac59d7299076fb98bdc5dea3722df8f90485ec1057531163\": container with ID starting with aa03008a9ce10d11ac59d7299076fb98bdc5dea3722df8f90485ec1057531163 not found: ID does not exist"
Nov 03 21:14:27.796323 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:27.796291    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main" probeResult="failure" output="Get \"https://10.133.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:14:27.829119 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:27.829054    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"]
Nov 03 21:14:27.878121 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:27.878098    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-862138/distinct-2-w2-862138-kserve-849dbc6564-rlh78"]
Nov 03 21:14:29.585469 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:29.585383    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" path="/var/lib/kubelet/pods/8fe2fef6-f3ea-49a2-aa27-47635359b017/volumes"
Nov 03 21:14:30.916956 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:30.916926    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"]
Nov 03 21:14:30.917362 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:30.917126    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm" podUID="85847bd0-1857-4ba7-b4c4-ada49fc94ae3" containerName="maas-api" containerID="cri-o://335e311dcc7b9c609407b6820499a740b4e04fc161130baa3670cf8b6ce7257d" gracePeriod=30
Nov 03 21:14:31.222681 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.222638    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 21:14:31.279873 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.279825    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-metrics-certs\") pod \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") "
Nov 03 21:14:31.280061 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.279921    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-kube-api-access-fqqfg\" (UniqueName: \"kubernetes.io/projected/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-kube-api-access-fqqfg\") pod \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") "
Nov 03 21:14:31.280061 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.280001    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-openshift-service-ca\") pod \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") "
Nov 03 21:14:31.280061 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.280044    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-maas-api-tls\") pod \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\" (UID: \"85847bd0-1857-4ba7-b4c4-ada49fc94ae3\") "
Nov 03 21:14:31.280453 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.280409    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-openshift-service-ca" pod "85847bd0-1857-4ba7-b4c4-ada49fc94ae3" (UID: "85847bd0-1857-4ba7-b4c4-ada49fc94ae3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:14:31.282555 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.282516    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-kube-api-access-fqqfg" pod "85847bd0-1857-4ba7-b4c4-ada49fc94ae3" (UID: "85847bd0-1857-4ba7-b4c4-ada49fc94ae3"). InnerVolumeSpecName "kube-api-access-fqqfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:31.282724 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.282691    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-metrics-certs" pod "85847bd0-1857-4ba7-b4c4-ada49fc94ae3" (UID: "85847bd0-1857-4ba7-b4c4-ada49fc94ae3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:31.283147 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.283118    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-maas-api-tls" pod "85847bd0-1857-4ba7-b4c4-ada49fc94ae3" (UID: "85847bd0-1857-4ba7-b4c4-ada49fc94ae3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:14:31.381530 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.381484    4993 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-metrics-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:31.381530 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.381516    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqqfg\" (UniqueName: \"kubernetes.io/projected/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-kube-api-access-fqqfg\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:31.381530 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.381526    4993 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-openshift-service-ca\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:31.381530 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.381535    4993 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85847bd0-1857-4ba7-b4c4-ada49fc94ae3-maas-api-tls\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:31.669930 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:14:31.669892    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:14:31.685454 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.685425    4993 generic.go:356] "Generic (PLEG): container finished" podID="85847bd0-1857-4ba7-b4c4-ada49fc94ae3" containerID="335e311dcc7b9c609407b6820499a740b4e04fc161130baa3670cf8b6ce7257d" exitCode=0
Nov 03 21:14:31.685617 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.685483    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"
Nov 03 21:14:31.685617 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.685509    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm" event={"ID":"85847bd0-1857-4ba7-b4c4-ada49fc94ae3","Type":"ContainerDied","Data":"335e311dcc7b9c609407b6820499a740b4e04fc161130baa3670cf8b6ce7257d"}
Nov 03 21:14:31.685617 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.685564    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm" event={"ID":"85847bd0-1857-4ba7-b4c4-ada49fc94ae3","Type":"ContainerDied","Data":"a8a8e4353ba8e48007ed622c44c48e6d548adab149959bb27b607764218821cd"}
Nov 03 21:14:31.685617 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.685588    4993 scope.go:122] "RemoveContainer" containerID="335e311dcc7b9c609407b6820499a740b4e04fc161130baa3670cf8b6ce7257d"
Nov 03 21:14:31.701010 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.700979    4993 scope.go:122] "RemoveContainer" containerID="335e311dcc7b9c609407b6820499a740b4e04fc161130baa3670cf8b6ce7257d"
Nov 03 21:14:31.701474 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:14:31.701444    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"335e311dcc7b9c609407b6820499a740b4e04fc161130baa3670cf8b6ce7257d\": container with ID starting with 335e311dcc7b9c609407b6820499a740b4e04fc161130baa3670cf8b6ce7257d not found: ID does not exist" containerID="335e311dcc7b9c609407b6820499a740b4e04fc161130baa3670cf8b6ce7257d"
Nov 03 21:14:31.701621 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.701594    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"335e311dcc7b9c609407b6820499a740b4e04fc161130baa3670cf8b6ce7257d"} err="failed to get container status \"335e311dcc7b9c609407b6820499a740b4e04fc161130baa3670cf8b6ce7257d\": rpc error: code = NotFound desc = could not find container \"335e311dcc7b9c609407b6820499a740b4e04fc161130baa3670cf8b6ce7257d\": container with ID starting with 335e311dcc7b9c609407b6820499a740b4e04fc161130baa3670cf8b6ce7257d not found: ID does not exist"
Nov 03 21:14:31.855069 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.854983    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"]
Nov 03 21:14:31.904609 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:31.904572    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-12a423-6c44f9546b-5lxjm"]
Nov 03 21:14:33.576088 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:33.576052    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85847bd0-1857-4ba7-b4c4-ada49fc94ae3" path="/var/lib/kubelet/pods/85847bd0-1857-4ba7-b4c4-ada49fc94ae3/volumes"
Nov 03 21:14:34.557271 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:34.557236    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"]
Nov 03 21:14:34.557526 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:34.557500    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79" podUID="96fbe5e3-4665-45ac-bc64-f128c312fb05" containerName="istio-proxy" containerID="cri-o://d30e47f236b33479dfd2e242fce60c5c09f7a576165a6629d9b8b231769ac323" gracePeriod=30
Nov 03 21:14:38.917499 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:38.917462    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"]
Nov 03 21:14:38.918011 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:38.917737    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94" podUID="a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" containerName="istio-proxy" containerID="cri-o://71538af0cfaddbaa3bf3cf6a7161b08fea108960c5f8984244cdc715f6426b22" gracePeriod=30
Nov 03 21:14:38.935455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:38.935419    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"]
Nov 03 21:14:38.935719 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:38.935691    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48" podUID="732f4b2d-0d2f-400d-8b19-38c237a6ecac" containerName="istio-proxy" containerID="cri-o://5755642b24d4b76bb7f641705089aefa0a166bbe35be2eb648fe43e545d667e5" gracePeriod=30
Nov 03 21:14:39.752734 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.752698    4993 generic.go:356] "Generic (PLEG): container finished" podID="96fbe5e3-4665-45ac-bc64-f128c312fb05" containerID="d30e47f236b33479dfd2e242fce60c5c09f7a576165a6629d9b8b231769ac323" exitCode=0
Nov 03 21:14:39.752897 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.752789    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79" event={"ID":"96fbe5e3-4665-45ac-bc64-f128c312fb05","Type":"ContainerDied","Data":"d30e47f236b33479dfd2e242fce60c5c09f7a576165a6629d9b8b231769ac323"}
Nov 03 21:14:39.752897 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.752837    4993 scope.go:122] "RemoveContainer" containerID="056409b81300dc4b86f1042405019bd6a73cdb77a134d8d96c2334a89d63ebcb"
Nov 03 21:14:39.824749 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.824689    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.901439    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-data\") pod \"96fbe5e3-4665-45ac-bc64-f128c312fb05\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") "
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.901554    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-credential-socket\") pod \"96fbe5e3-4665-45ac-bc64-f128c312fb05\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") "
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.901659    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-podinfo\") pod \"96fbe5e3-4665-45ac-bc64-f128c312fb05\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") "
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.901714    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96fbe5e3-4665-45ac-bc64-f128c312fb05-kube-api-access-dmnj6\" (UniqueName: \"kubernetes.io/projected/96fbe5e3-4665-45ac-bc64-f128c312fb05-kube-api-access-dmnj6\") pod \"96fbe5e3-4665-45ac-bc64-f128c312fb05\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") "
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.901789    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-token\" (UniqueName: \"kubernetes.io/projected/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-token\") pod \"96fbe5e3-4665-45ac-bc64-f128c312fb05\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") "
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.901815    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-data" pod "96fbe5e3-4665-45ac-bc64-f128c312fb05" (UID: "96fbe5e3-4665-45ac-bc64-f128c312fb05"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.901841    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-workload-socket\") pod \"96fbe5e3-4665-45ac-bc64-f128c312fb05\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") "
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.901903    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96fbe5e3-4665-45ac-bc64-f128c312fb05-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96fbe5e3-4665-45ac-bc64-f128c312fb05-istiod-ca-cert\") pod \"96fbe5e3-4665-45ac-bc64-f128c312fb05\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") "
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.901990    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-envoy\") pod \"96fbe5e3-4665-45ac-bc64-f128c312fb05\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") "
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.902034    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-workload-certs\") pod \"96fbe5e3-4665-45ac-bc64-f128c312fb05\" (UID: \"96fbe5e3-4665-45ac-bc64-f128c312fb05\") "
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.902317    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-credential-socket" pod "96fbe5e3-4665-45ac-bc64-f128c312fb05" (UID: "96fbe5e3-4665-45ac-bc64-f128c312fb05"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.902448    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.902466    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.902592    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-workload-certs" pod "96fbe5e3-4665-45ac-bc64-f128c312fb05" (UID: "96fbe5e3-4665-45ac-bc64-f128c312fb05"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.902691    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-workload-socket" pod "96fbe5e3-4665-45ac-bc64-f128c312fb05" (UID: "96fbe5e3-4665-45ac-bc64-f128c312fb05"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:39.903536 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.902972    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96fbe5e3-4665-45ac-bc64-f128c312fb05-istiod-ca-cert" pod "96fbe5e3-4665-45ac-bc64-f128c312fb05" (UID: "96fbe5e3-4665-45ac-bc64-f128c312fb05"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:14:39.905949 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.905922    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-podinfo" pod "96fbe5e3-4665-45ac-bc64-f128c312fb05" (UID: "96fbe5e3-4665-45ac-bc64-f128c312fb05"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:14:39.905949 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.905942    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-envoy" pod "96fbe5e3-4665-45ac-bc64-f128c312fb05" (UID: "96fbe5e3-4665-45ac-bc64-f128c312fb05"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:39.906144 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.905949    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-token" pod "96fbe5e3-4665-45ac-bc64-f128c312fb05" (UID: "96fbe5e3-4665-45ac-bc64-f128c312fb05"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:39.906144 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:39.905984    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96fbe5e3-4665-45ac-bc64-f128c312fb05-kube-api-access-dmnj6" pod "96fbe5e3-4665-45ac-bc64-f128c312fb05" (UID: "96fbe5e3-4665-45ac-bc64-f128c312fb05"). InnerVolumeSpecName "kube-api-access-dmnj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:40.003601 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:40.003567    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:40.003601 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:40.003596    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96fbe5e3-4665-45ac-bc64-f128c312fb05-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:40.004029 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:40.003611    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:40.004029 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:40.003623    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96fbe5e3-4665-45ac-bc64-f128c312fb05-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:40.004029 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:40.003663    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:40.004029 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:40.003677    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmnj6\" (UniqueName: \"kubernetes.io/projected/96fbe5e3-4665-45ac-bc64-f128c312fb05-kube-api-access-dmnj6\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:40.004029 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:40.003691    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96fbe5e3-4665-45ac-bc64-f128c312fb05-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:40.760102 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:40.760067    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79" event={"ID":"96fbe5e3-4665-45ac-bc64-f128c312fb05","Type":"ContainerDied","Data":"2994421c913b40fa9acfcceda9ecff5825168a15f44026dbc4da44a3d339e675"}
Nov 03 21:14:40.760102 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:40.760086    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"
Nov 03 21:14:40.760332 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:40.760135    4993 scope.go:122] "RemoveContainer" containerID="d30e47f236b33479dfd2e242fce60c5c09f7a576165a6629d9b8b231769ac323"
Nov 03 21:14:40.946766 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:40.946730    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"]
Nov 03 21:14:40.946970 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:40.946946    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g" podUID="f9b3af9c-bf26-4963-9146-7dadd2eb20ba" containerName="istio-proxy" containerID="cri-o://39c6b389d43c8145088ed71b2454988e3180da2a7ec25af24b83b89e30d2481e" gracePeriod=30
Nov 03 21:14:41.033124 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:41.033018    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"]
Nov 03 21:14:41.035195 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:41.035154    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-e22abf-openshift-default-5f77dbbb7-9zq79"]
Nov 03 21:14:41.578306 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:41.578266    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96fbe5e3-4665-45ac-bc64-f128c312fb05" path="/var/lib/kubelet/pods/96fbe5e3-4665-45ac-bc64-f128c312fb05/volumes"
Nov 03 21:14:41.863702 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:41.862864    4993 patch_prober.go:29] interesting pod/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:14:41.863702 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:41.862936    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g" podUID="f9b3af9c-bf26-4963-9146-7dadd2eb20ba" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:14:43.151311 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:43.151268    4993 patch_prober.go:29] interesting pod/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 03 21:14:43.151777 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:43.151328    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48" podUID="732f4b2d-0d2f-400d-8b19-38c237a6ecac" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 03 21:14:44.191856 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.191830    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 21:14:44.222605 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.222584    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 21:14:44.348296 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348261    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-data\") pod \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") "
Nov 03 21:14:44.348449 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348311    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-workload-certs\") pod \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") "
Nov 03 21:14:44.348449 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348360    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istiod-ca-cert\") pod \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") "
Nov 03 21:14:44.348449 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348387    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istiod-ca-cert\") pod \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") "
Nov 03 21:14:44.348449 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348419    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-envoy\") pod \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") "
Nov 03 21:14:44.348685 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348450    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-workload-socket\") pod \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") "
Nov 03 21:14:44.348685 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348492    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-credential-socket\") pod \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") "
Nov 03 21:14:44.348685 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348525    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-token\" (UniqueName: \"kubernetes.io/projected/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-token\") pod \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") "
Nov 03 21:14:44.348685 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348554    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-token\" (UniqueName: \"kubernetes.io/projected/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-token\") pod \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") "
Nov 03 21:14:44.348685 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348585    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-kube-api-access-rl9km\" (UniqueName: \"kubernetes.io/projected/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-kube-api-access-rl9km\") pod \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") "
Nov 03 21:14:44.348685 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348595    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-data" pod "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" (UID: "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:44.348685 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348614    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/732f4b2d-0d2f-400d-8b19-38c237a6ecac-kube-api-access-g5kgj\" (UniqueName: \"kubernetes.io/projected/732f4b2d-0d2f-400d-8b19-38c237a6ecac-kube-api-access-g5kgj\") pod \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") "
Nov 03 21:14:44.348685 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348663    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-workload-certs\") pod \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") "
Nov 03 21:14:44.349028 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348708    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-workload-socket\") pod \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") "
Nov 03 21:14:44.349028 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348733    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-credential-socket\") pod \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") "
Nov 03 21:14:44.349028 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.348794    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-envoy\") pod \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") "
Nov 03 21:14:44.349325 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.349290    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-workload-certs" pod "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" (UID: "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:44.349463 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.349320    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-workload-socket" pod "732f4b2d-0d2f-400d-8b19-38c237a6ecac" (UID: "732f4b2d-0d2f-400d-8b19-38c237a6ecac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:44.349529 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.349455    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-podinfo\") pod \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") "
Nov 03 21:14:44.349529 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.349517    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-podinfo\") pod \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\" (UID: \"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6\") "
Nov 03 21:14:44.349662 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.349539    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-credential-socket" pod "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" (UID: "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:44.349662 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.349574    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-data\") pod \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\" (UID: \"732f4b2d-0d2f-400d-8b19-38c237a6ecac\") "
Nov 03 21:14:44.349662 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.349599    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-credential-socket" pod "732f4b2d-0d2f-400d-8b19-38c237a6ecac" (UID: "732f4b2d-0d2f-400d-8b19-38c237a6ecac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:44.349812 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.349764    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-workload-certs" pod "732f4b2d-0d2f-400d-8b19-38c237a6ecac" (UID: "732f4b2d-0d2f-400d-8b19-38c237a6ecac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:44.350037 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.349974    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istiod-ca-cert" pod "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" (UID: "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:14:44.350137 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.350082    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.350137 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.350091    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-data" pod "732f4b2d-0d2f-400d-8b19-38c237a6ecac" (UID: "732f4b2d-0d2f-400d-8b19-38c237a6ecac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:44.350137 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.350101    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.350303 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.350150    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.350303 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.350170    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.350303 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.350185    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.350303 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.350199    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.350303 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.350212    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.352143 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.352108    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-envoy" pod "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" (UID: "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:44.352260 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.352171    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-envoy" pod "732f4b2d-0d2f-400d-8b19-38c237a6ecac" (UID: "732f4b2d-0d2f-400d-8b19-38c237a6ecac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:44.352561 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.352484    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/732f4b2d-0d2f-400d-8b19-38c237a6ecac-kube-api-access-g5kgj" pod "732f4b2d-0d2f-400d-8b19-38c237a6ecac" (UID: "732f4b2d-0d2f-400d-8b19-38c237a6ecac"). InnerVolumeSpecName "kube-api-access-g5kgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:44.352815 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.352791    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-token" pod "732f4b2d-0d2f-400d-8b19-38c237a6ecac" (UID: "732f4b2d-0d2f-400d-8b19-38c237a6ecac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:44.353374 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.353340    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-podinfo" pod "732f4b2d-0d2f-400d-8b19-38c237a6ecac" (UID: "732f4b2d-0d2f-400d-8b19-38c237a6ecac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:14:44.353523 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.353505    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-token" pod "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" (UID: "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:44.353696 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.353669    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istiod-ca-cert" pod "732f4b2d-0d2f-400d-8b19-38c237a6ecac" (UID: "732f4b2d-0d2f-400d-8b19-38c237a6ecac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:14:44.353790 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.353714    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-kube-api-access-rl9km" pod "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" (UID: "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6"). InnerVolumeSpecName "kube-api-access-rl9km". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:44.353915 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.353893    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-workload-socket" pod "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" (UID: "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:44.354339 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.354314    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-podinfo" pod "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" (UID: "a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:14:44.450873 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.450845    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.450873 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.450871    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.451067 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.450882    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rl9km\" (UniqueName: \"kubernetes.io/projected/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-kube-api-access-rl9km\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.451067 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.450892    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5kgj\" (UniqueName: \"kubernetes.io/projected/732f4b2d-0d2f-400d-8b19-38c237a6ecac-kube-api-access-g5kgj\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.451067 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.450900    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.451067 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.450908    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.451067 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.450919    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.451067 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.450930    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.451067 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.450938    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.451067 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.450947    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.451067 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.450955    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/732f4b2d-0d2f-400d-8b19-38c237a6ecac-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:44.783326 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.783287    4993 generic.go:356] "Generic (PLEG): container finished" podID="732f4b2d-0d2f-400d-8b19-38c237a6ecac" containerID="5755642b24d4b76bb7f641705089aefa0a166bbe35be2eb648fe43e545d667e5" exitCode=0
Nov 03 21:14:44.783490 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.783341    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"
Nov 03 21:14:44.783490 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.783371    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48" event={"ID":"732f4b2d-0d2f-400d-8b19-38c237a6ecac","Type":"ContainerDied","Data":"5755642b24d4b76bb7f641705089aefa0a166bbe35be2eb648fe43e545d667e5"}
Nov 03 21:14:44.783490 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.783425    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48" event={"ID":"732f4b2d-0d2f-400d-8b19-38c237a6ecac","Type":"ContainerDied","Data":"a7f9dc83097c556d08e0999b470a2bf7c8faa0b4ccbf765b0858d7b481ac67ee"}
Nov 03 21:14:44.783490 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.783446    4993 scope.go:122] "RemoveContainer" containerID="5755642b24d4b76bb7f641705089aefa0a166bbe35be2eb648fe43e545d667e5"
Nov 03 21:14:44.785163 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.785138    4993 generic.go:356] "Generic (PLEG): container finished" podID="a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" containerID="71538af0cfaddbaa3bf3cf6a7161b08fea108960c5f8984244cdc715f6426b22" exitCode=0
Nov 03 21:14:44.785273 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.785180    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"
Nov 03 21:14:44.785273 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.785195    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94" event={"ID":"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6","Type":"ContainerDied","Data":"71538af0cfaddbaa3bf3cf6a7161b08fea108960c5f8984244cdc715f6426b22"}
Nov 03 21:14:44.785273 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.785234    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94" event={"ID":"a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6","Type":"ContainerDied","Data":"2af52f060eb52efdcb702e7625fa4c9a6df80abe1720877e68e4f03676d4f14e"}
Nov 03 21:14:44.793347 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.793329    4993 scope.go:122] "RemoveContainer" containerID="5755642b24d4b76bb7f641705089aefa0a166bbe35be2eb648fe43e545d667e5"
Nov 03 21:14:44.793767 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:14:44.793610    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5755642b24d4b76bb7f641705089aefa0a166bbe35be2eb648fe43e545d667e5\": container with ID starting with 5755642b24d4b76bb7f641705089aefa0a166bbe35be2eb648fe43e545d667e5 not found: ID does not exist" containerID="5755642b24d4b76bb7f641705089aefa0a166bbe35be2eb648fe43e545d667e5"
Nov 03 21:14:44.793767 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.793674    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5755642b24d4b76bb7f641705089aefa0a166bbe35be2eb648fe43e545d667e5"} err="failed to get container status \"5755642b24d4b76bb7f641705089aefa0a166bbe35be2eb648fe43e545d667e5\": rpc error: code = NotFound desc = could not find container \"5755642b24d4b76bb7f641705089aefa0a166bbe35be2eb648fe43e545d667e5\": container with ID starting with 5755642b24d4b76bb7f641705089aefa0a166bbe35be2eb648fe43e545d667e5 not found: ID does not exist"
Nov 03 21:14:44.793767 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.793694    4993 scope.go:122] "RemoveContainer" containerID="71538af0cfaddbaa3bf3cf6a7161b08fea108960c5f8984244cdc715f6426b22"
Nov 03 21:14:44.802192 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.802172    4993 scope.go:122] "RemoveContainer" containerID="71538af0cfaddbaa3bf3cf6a7161b08fea108960c5f8984244cdc715f6426b22"
Nov 03 21:14:44.802459 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:14:44.802441    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71538af0cfaddbaa3bf3cf6a7161b08fea108960c5f8984244cdc715f6426b22\": container with ID starting with 71538af0cfaddbaa3bf3cf6a7161b08fea108960c5f8984244cdc715f6426b22 not found: ID does not exist" containerID="71538af0cfaddbaa3bf3cf6a7161b08fea108960c5f8984244cdc715f6426b22"
Nov 03 21:14:44.802514 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.802471    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71538af0cfaddbaa3bf3cf6a7161b08fea108960c5f8984244cdc715f6426b22"} err="failed to get container status \"71538af0cfaddbaa3bf3cf6a7161b08fea108960c5f8984244cdc715f6426b22\": rpc error: code = NotFound desc = could not find container \"71538af0cfaddbaa3bf3cf6a7161b08fea108960c5f8984244cdc715f6426b22\": container with ID starting with 71538af0cfaddbaa3bf3cf6a7161b08fea108960c5f8984244cdc715f6426b22 not found: ID does not exist"
Nov 03 21:14:44.951435 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.951404    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"]
Nov 03 21:14:44.999976 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:44.999942    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-6ps94"]
Nov 03 21:14:45.065864 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:45.065795    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"]
Nov 03 21:14:45.070042 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:45.070019    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-862138-openshift-default-bc56c55fc-9bh48"]
Nov 03 21:14:45.577328 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:45.577297    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="732f4b2d-0d2f-400d-8b19-38c237a6ecac" path="/var/lib/kubelet/pods/732f4b2d-0d2f-400d-8b19-38c237a6ecac/volumes"
Nov 03 21:14:45.577766 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:45.577752    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" path="/var/lib/kubelet/pods/a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6/volumes"
Nov 03 21:14:46.200893 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.200870    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 21:14:46.367366 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.367282    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-envoy\") pod \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") "
Nov 03 21:14:46.367366 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.367323    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-data\") pod \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") "
Nov 03 21:14:46.367604 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.367366    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-workload-socket\") pod \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") "
Nov 03 21:14:46.367604 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.367403    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-podinfo\") pod \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") "
Nov 03 21:14:46.367604 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.367427    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-token\" (UniqueName: \"kubernetes.io/projected/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-token\") pod \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") "
Nov 03 21:14:46.367604 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.367555    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-kube-api-access-st5gt\" (UniqueName: \"kubernetes.io/projected/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-kube-api-access-st5gt\") pod \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") "
Nov 03 21:14:46.367604 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.367603    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istiod-ca-cert\") pod \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") "
Nov 03 21:14:46.367906 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.367660    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-credential-socket\") pod \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") "
Nov 03 21:14:46.367906 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.367710    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-workload-certs\") pod \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\" (UID: \"f9b3af9c-bf26-4963-9146-7dadd2eb20ba\") "
Nov 03 21:14:46.367906 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.367713    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-data" pod "f9b3af9c-bf26-4963-9146-7dadd2eb20ba" (UID: "f9b3af9c-bf26-4963-9146-7dadd2eb20ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:46.367906 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.367750    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-workload-socket" pod "f9b3af9c-bf26-4963-9146-7dadd2eb20ba" (UID: "f9b3af9c-bf26-4963-9146-7dadd2eb20ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:46.368108 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.368060    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.368108 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.368071    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-workload-certs" pod "f9b3af9c-bf26-4963-9146-7dadd2eb20ba" (UID: "f9b3af9c-bf26-4963-9146-7dadd2eb20ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:46.368108 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.368083    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istiod-ca-cert" pod "f9b3af9c-bf26-4963-9146-7dadd2eb20ba" (UID: "f9b3af9c-bf26-4963-9146-7dadd2eb20ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:14:46.368108 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.368081    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.368289 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.368238    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-credential-socket" pod "f9b3af9c-bf26-4963-9146-7dadd2eb20ba" (UID: "f9b3af9c-bf26-4963-9146-7dadd2eb20ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:46.370251 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.370159    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-kube-api-access-st5gt" pod "f9b3af9c-bf26-4963-9146-7dadd2eb20ba" (UID: "f9b3af9c-bf26-4963-9146-7dadd2eb20ba"). InnerVolumeSpecName "kube-api-access-st5gt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:46.370566 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.370406    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-podinfo" pod "f9b3af9c-bf26-4963-9146-7dadd2eb20ba" (UID: "f9b3af9c-bf26-4963-9146-7dadd2eb20ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:14:46.370704 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.370571    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-envoy" pod "f9b3af9c-bf26-4963-9146-7dadd2eb20ba" (UID: "f9b3af9c-bf26-4963-9146-7dadd2eb20ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:14:46.371797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.371775    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-token" pod "f9b3af9c-bf26-4963-9146-7dadd2eb20ba" (UID: "f9b3af9c-bf26-4963-9146-7dadd2eb20ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:14:46.468543 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.468511    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.468543 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.468542    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.468783 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.468555    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.468783 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.468568    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-st5gt\" (UniqueName: \"kubernetes.io/projected/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-kube-api-access-st5gt\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.468783 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.468581    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.468783 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.468592    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.468783 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.468605    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9b3af9c-bf26-4963-9146-7dadd2eb20ba-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:14:46.799886 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.799852    4993 generic.go:356] "Generic (PLEG): container finished" podID="f9b3af9c-bf26-4963-9146-7dadd2eb20ba" containerID="39c6b389d43c8145088ed71b2454988e3180da2a7ec25af24b83b89e30d2481e" exitCode=0
Nov 03 21:14:46.800280 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.799899    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"
Nov 03 21:14:46.800280 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.799930    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g" event={"ID":"f9b3af9c-bf26-4963-9146-7dadd2eb20ba","Type":"ContainerDied","Data":"39c6b389d43c8145088ed71b2454988e3180da2a7ec25af24b83b89e30d2481e"}
Nov 03 21:14:46.800280 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.799989    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g" event={"ID":"f9b3af9c-bf26-4963-9146-7dadd2eb20ba","Type":"ContainerDied","Data":"da4e23b111682eb8eaf9c64e96ddeadbc160edc4513a4f2a364afda7982deebe"}
Nov 03 21:14:46.800280 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.800012    4993 scope.go:122] "RemoveContainer" containerID="39c6b389d43c8145088ed71b2454988e3180da2a7ec25af24b83b89e30d2481e"
Nov 03 21:14:46.809436 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.809420    4993 scope.go:122] "RemoveContainer" containerID="39c6b389d43c8145088ed71b2454988e3180da2a7ec25af24b83b89e30d2481e"
Nov 03 21:14:46.809782 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:14:46.809751    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39c6b389d43c8145088ed71b2454988e3180da2a7ec25af24b83b89e30d2481e\": container with ID starting with 39c6b389d43c8145088ed71b2454988e3180da2a7ec25af24b83b89e30d2481e not found: ID does not exist" containerID="39c6b389d43c8145088ed71b2454988e3180da2a7ec25af24b83b89e30d2481e"
Nov 03 21:14:46.809855 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.809790    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39c6b389d43c8145088ed71b2454988e3180da2a7ec25af24b83b89e30d2481e"} err="failed to get container status \"39c6b389d43c8145088ed71b2454988e3180da2a7ec25af24b83b89e30d2481e\": rpc error: code = NotFound desc = could not find container \"39c6b389d43c8145088ed71b2454988e3180da2a7ec25af24b83b89e30d2481e\": container with ID starting with 39c6b389d43c8145088ed71b2454988e3180da2a7ec25af24b83b89e30d2481e not found: ID does not exist"
Nov 03 21:14:46.966052 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:46.966014    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"]
Nov 03 21:14:47.015824 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:47.015791    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-12a423-openshift-default-747695c87f-8s29g"]
Nov 03 21:14:47.575873 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:14:47.575843    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9b3af9c-bf26-4963-9146-7dadd2eb20ba" path="/var/lib/kubelet/pods/f9b3af9c-bf26-4963-9146-7dadd2eb20ba/volumes"
Nov 03 21:15:01.668985 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:15:01.668939    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:15:19.158405 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.158370    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"]
Nov 03 21:15:19.158943 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.158599    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk" podUID="293bb466-4d73-49bf-aa53-cf512e226e5b" containerName="discovery" containerID="cri-o://212c89824de35072bb7651ba1a6d89da153d215f3b4350ff4a7bc6d711257772" gracePeriod=30
Nov 03 21:15:19.412752 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.412690    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:15:19.599589 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.599557    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-csr-dns-cert\") pod \"293bb466-4d73-49bf-aa53-cf512e226e5b\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") "
Nov 03 21:15:19.599798 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.599615    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-kubeconfig\") pod \"293bb466-4d73-49bf-aa53-cf512e226e5b\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") "
Nov 03 21:15:19.599798 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.599670    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/293bb466-4d73-49bf-aa53-cf512e226e5b-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/293bb466-4d73-49bf-aa53-cf512e226e5b-local-certs\") pod \"293bb466-4d73-49bf-aa53-cf512e226e5b\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") "
Nov 03 21:15:19.599798 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.599723    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-token\" (UniqueName: \"kubernetes.io/projected/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-token\") pod \"293bb466-4d73-49bf-aa53-cf512e226e5b\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") "
Nov 03 21:15:19.599798 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.599779    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-cacerts\" (UniqueName: \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-cacerts\") pod \"293bb466-4d73-49bf-aa53-cf512e226e5b\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") "
Nov 03 21:15:19.600049 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.599826    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/293bb466-4d73-49bf-aa53-cf512e226e5b-kube-api-access-ccv9m\" (UniqueName: \"kubernetes.io/projected/293bb466-4d73-49bf-aa53-cf512e226e5b-kube-api-access-ccv9m\") pod \"293bb466-4d73-49bf-aa53-cf512e226e5b\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") "
Nov 03 21:15:19.600049 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.599872    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-csr-ca-configmap\") pod \"293bb466-4d73-49bf-aa53-cf512e226e5b\" (UID: \"293bb466-4d73-49bf-aa53-cf512e226e5b\") "
Nov 03 21:15:19.600764 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.600735    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-csr-ca-configmap" pod "293bb466-4d73-49bf-aa53-cf512e226e5b" (UID: "293bb466-4d73-49bf-aa53-cf512e226e5b"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:15:19.602523 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.602488    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-kubeconfig" pod "293bb466-4d73-49bf-aa53-cf512e226e5b" (UID: "293bb466-4d73-49bf-aa53-cf512e226e5b"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:15:19.602694 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.602671    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-token" pod "293bb466-4d73-49bf-aa53-cf512e226e5b" (UID: "293bb466-4d73-49bf-aa53-cf512e226e5b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:15:19.602776 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.602699    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293bb466-4d73-49bf-aa53-cf512e226e5b-local-certs" pod "293bb466-4d73-49bf-aa53-cf512e226e5b" (UID: "293bb466-4d73-49bf-aa53-cf512e226e5b"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:15:19.602776 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.602707    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-csr-dns-cert" pod "293bb466-4d73-49bf-aa53-cf512e226e5b" (UID: "293bb466-4d73-49bf-aa53-cf512e226e5b"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:15:19.602906 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.602852    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/293bb466-4d73-49bf-aa53-cf512e226e5b-kube-api-access-ccv9m" pod "293bb466-4d73-49bf-aa53-cf512e226e5b" (UID: "293bb466-4d73-49bf-aa53-cf512e226e5b"). InnerVolumeSpecName "kube-api-access-ccv9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:15:19.603144 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.603118    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-cacerts" pod "293bb466-4d73-49bf-aa53-cf512e226e5b" (UID: "293bb466-4d73-49bf-aa53-cf512e226e5b"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:15:19.701073 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.700981    4993 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-kubeconfig\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:15:19.701073 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.701007    4993 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/293bb466-4d73-49bf-aa53-cf512e226e5b-local-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:15:19.701073 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.701016    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:15:19.701073 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.701025    4993 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-cacerts\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:15:19.701073 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.701035    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccv9m\" (UniqueName: \"kubernetes.io/projected/293bb466-4d73-49bf-aa53-cf512e226e5b-kube-api-access-ccv9m\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:15:19.701073 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.701045    4993 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-csr-ca-configmap\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:15:19.701073 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.701058    4993 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/293bb466-4d73-49bf-aa53-cf512e226e5b-istio-csr-dns-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:15:19.960174 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.960137    4993 generic.go:356] "Generic (PLEG): container finished" podID="293bb466-4d73-49bf-aa53-cf512e226e5b" containerID="212c89824de35072bb7651ba1a6d89da153d215f3b4350ff4a7bc6d711257772" exitCode=0
Nov 03 21:15:19.960334 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.960183    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"
Nov 03 21:15:19.960334 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.960223    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk" event={"ID":"293bb466-4d73-49bf-aa53-cf512e226e5b","Type":"ContainerDied","Data":"212c89824de35072bb7651ba1a6d89da153d215f3b4350ff4a7bc6d711257772"}
Nov 03 21:15:19.960334 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.960292    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk" event={"ID":"293bb466-4d73-49bf-aa53-cf512e226e5b","Type":"ContainerDied","Data":"eaaf69050efb2b252764ba6f2b62871bb8b81586410bcd3d5f9d0a7d10cea525"}
Nov 03 21:15:19.960334 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.960321    4993 scope.go:122] "RemoveContainer" containerID="212c89824de35072bb7651ba1a6d89da153d215f3b4350ff4a7bc6d711257772"
Nov 03 21:15:19.970951 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.970935    4993 scope.go:122] "RemoveContainer" containerID="212c89824de35072bb7651ba1a6d89da153d215f3b4350ff4a7bc6d711257772"
Nov 03 21:15:19.971263 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:15:19.971245    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"212c89824de35072bb7651ba1a6d89da153d215f3b4350ff4a7bc6d711257772\": container with ID starting with 212c89824de35072bb7651ba1a6d89da153d215f3b4350ff4a7bc6d711257772 not found: ID does not exist" containerID="212c89824de35072bb7651ba1a6d89da153d215f3b4350ff4a7bc6d711257772"
Nov 03 21:15:19.971321 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:19.971271    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"212c89824de35072bb7651ba1a6d89da153d215f3b4350ff4a7bc6d711257772"} err="failed to get container status \"212c89824de35072bb7651ba1a6d89da153d215f3b4350ff4a7bc6d711257772\": rpc error: code = NotFound desc = could not find container \"212c89824de35072bb7651ba1a6d89da153d215f3b4350ff4a7bc6d711257772\": container with ID starting with 212c89824de35072bb7651ba1a6d89da153d215f3b4350ff4a7bc6d711257772 not found: ID does not exist"
Nov 03 21:15:20.173864 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:20.173829    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"]
Nov 03 21:15:20.176833 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:20.176809    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-56nxk"]
Nov 03 21:15:21.575896 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:15:21.575859    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="293bb466-4d73-49bf-aa53-cf512e226e5b" path="/var/lib/kubelet/pods/293bb466-4d73-49bf-aa53-cf512e226e5b/volumes"
Nov 03 21:15:31.670193 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:15:31.670158    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:16:01.670886 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:16:01.670850    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:16:11.998391 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:16:11.997730    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:16:11.998391 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:16:11.997793    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-cws5p" podUID="6bf4e14e-9896-452a-bfe9-58e3f89a155c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:16:31.670032 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:16:31.669995    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:17:01.669043 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:17:01.669007    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:17:31.668938 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:17:31.668897    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:18:01.669744 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:18:01.669658    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:18:31.670477 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:18:31.670443    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:19:01.669297 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:19:01.669253    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:19:31.669549 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:19:31.669504    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:20:01.669488 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:20:01.669441    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:20:31.670410 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:20:31.670370    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:21:01.668569 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:21:01.668488    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:21:31.669270 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:21:31.669231    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:22:01.669472 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:22:01.669355    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:22:31.673295 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:22:31.673249    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:22:42.276834 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.276799    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"]
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277163    4993 cpu_manager.go:414] "containerMap: removing container" podUID="f9b3af9c-bf26-4963-9146-7dadd2eb20ba" containerName="istio-proxy"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277174    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9b3af9c-bf26-4963-9146-7dadd2eb20ba" containerName="istio-proxy"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277186    4993 cpu_manager.go:414] "containerMap: removing container" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277192    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277205    4993 cpu_manager.go:414] "containerMap: removing container" podUID="529e3166-0d8e-40e1-93a7-a6b563ce5c8c" containerName="istio-proxy"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277210    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="529e3166-0d8e-40e1-93a7-a6b563ce5c8c" containerName="istio-proxy"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277221    4993 cpu_manager.go:414] "containerMap: removing container" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277225    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277231    4993 cpu_manager.go:414] "containerMap: removing container" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277236    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277242    4993 cpu_manager.go:414] "containerMap: removing container" podUID="a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" containerName="istio-proxy"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277246    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" containerName="istio-proxy"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277252    4993 cpu_manager.go:414] "containerMap: removing container" podUID="293bb466-4d73-49bf-aa53-cf512e226e5b" containerName="discovery"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277257    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="293bb466-4d73-49bf-aa53-cf512e226e5b" containerName="discovery"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277265    4993 cpu_manager.go:414] "containerMap: removing container" podUID="732f4b2d-0d2f-400d-8b19-38c237a6ecac" containerName="istio-proxy"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277269    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="732f4b2d-0d2f-400d-8b19-38c237a6ecac" containerName="istio-proxy"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277280    4993 cpu_manager.go:414] "containerMap: removing container" podUID="45229f7a-8c58-4b55-b532-ff4aaf5032dd" containerName="istio-proxy"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277284    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="45229f7a-8c58-4b55-b532-ff4aaf5032dd" containerName="istio-proxy"
Nov 03 21:22:42.277284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277291    4993 cpu_manager.go:414] "containerMap: removing container" podUID="96fbe5e3-4665-45ac-bc64-f128c312fb05" containerName="istio-proxy"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277295    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96fbe5e3-4665-45ac-bc64-f128c312fb05" containerName="istio-proxy"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277300    4993 cpu_manager.go:414] "containerMap: removing container" podUID="93aa090e-efe6-4953-8368-cb2b7bcdc394" containerName="test-curl-7364-d5cebd"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277305    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93aa090e-efe6-4953-8368-cb2b7bcdc394" containerName="test-curl-7364-d5cebd"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277310    4993 cpu_manager.go:414] "containerMap: removing container" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277314    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277326    4993 cpu_manager.go:414] "containerMap: removing container" podUID="85847bd0-1857-4ba7-b4c4-ada49fc94ae3" containerName="maas-api"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277330    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85847bd0-1857-4ba7-b4c4-ada49fc94ae3" containerName="maas-api"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277336    4993 cpu_manager.go:414] "containerMap: removing container" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277341    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277347    4993 cpu_manager.go:414] "containerMap: removing container" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277351    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277359    4993 cpu_manager.go:414] "containerMap: removing container" podUID="68cccb32-d2a6-48f7-98f6-f0f60aab5496" containerName="istio-proxy"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277363    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68cccb32-d2a6-48f7-98f6-f0f60aab5496" containerName="istio-proxy"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277373    4993 cpu_manager.go:414] "containerMap: removing container" podUID="dcdc5f37-f37d-43bc-8fea-ad101a050eec" containerName="maas-api"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277376    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dcdc5f37-f37d-43bc-8fea-ad101a050eec" containerName="maas-api"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277382    4993 cpu_manager.go:414] "containerMap: removing container" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277387    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277433    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ff97541-97f9-4ff6-b218-c717565ef6a4" containerName="main"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277440    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="529e3166-0d8e-40e1-93a7-a6b563ce5c8c" containerName="istio-proxy"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277446    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="81f1a530-ef53-4782-aae4-4a8666859325" containerName="main"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277452    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="b64da81f-0a97-4cdd-9afd-1d58a306c717" containerName="main"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277457    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9e8cb2e-098a-4cd1-8d7a-af05fdcd6fa6" containerName="istio-proxy"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277462    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="293bb466-4d73-49bf-aa53-cf512e226e5b" containerName="discovery"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277468    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="732f4b2d-0d2f-400d-8b19-38c237a6ecac" containerName="istio-proxy"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277474    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="45229f7a-8c58-4b55-b532-ff4aaf5032dd" containerName="istio-proxy"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277479    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="96fbe5e3-4665-45ac-bc64-f128c312fb05" containerName="istio-proxy"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277484    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="93aa090e-efe6-4953-8368-cb2b7bcdc394" containerName="test-curl-7364-d5cebd"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277489    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0c71be3-4788-4e30-8bfb-905e950a6d0a" containerName="main"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277495    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="85847bd0-1857-4ba7-b4c4-ada49fc94ae3" containerName="maas-api"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277500    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="809df5f7-98f7-4dbf-8548-b0f7dade9c40" containerName="main"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277505    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="8fe2fef6-f3ea-49a2-aa27-47635359b017" containerName="main"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277510    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="96fbe5e3-4665-45ac-bc64-f128c312fb05" containerName="istio-proxy"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277515    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="68cccb32-d2a6-48f7-98f6-f0f60aab5496" containerName="istio-proxy"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277522    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="dcdc5f37-f37d-43bc-8fea-ad101a050eec" containerName="maas-api"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277526    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="2204dd40-a737-47fc-a685-420d5b819a29" containerName="main"
Nov 03 21:22:42.277876 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.277530    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9b3af9c-bf26-4963-9146-7dadd2eb20ba" containerName="istio-proxy"
Nov 03 21:22:42.280474 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.280458    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:42.332753 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.332725    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-96c0ab-serving-cert\""
Nov 03 21:22:42.367871 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.367836    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8136baeb-6550-4400-866d-4796aa76ad35-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:42.368029 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.367885    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-metrics-certs\") pod \"maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:42.368029 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.368007    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64k7n\" (UniqueName: \"kubernetes.io/projected/8136baeb-6550-4400-866d-4796aa76ad35-kube-api-access-64k7n\") pod \"maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:42.368148 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.368095    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:42.383599 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.383573    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"]
Nov 03 21:22:42.468960 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.468930    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:42.469141 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.468995    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8136baeb-6550-4400-866d-4796aa76ad35-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:42.469141 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.469021    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-metrics-certs\") pod \"maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:42.469141 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:22:42.469091    4993 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-serving-cert: secret "maas-api-e2e-worker-single-main-96c0ab-serving-cert" not found
Nov 03 21:22:42.469317 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:22:42.469164    4993 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-maas-api-tls podName:8136baeb-6550-4400-866d-4796aa76ad35 nodeName:}" failed. No retries permitted until 2026-11-03 21:22:42.969146804 +0000 UTC m=+2565.513853156 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-maas-api-tls") pod "maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj" (UID: "8136baeb-6550-4400-866d-4796aa76ad35") : secret "maas-api-e2e-worker-single-main-96c0ab-serving-cert" not found
Nov 03 21:22:42.469317 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.469186    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64k7n\" (UniqueName: \"kubernetes.io/projected/8136baeb-6550-4400-866d-4796aa76ad35-kube-api-access-64k7n\") pod \"maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:42.469740 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.469720    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8136baeb-6550-4400-866d-4796aa76ad35-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:42.471537 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.471509    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-metrics-certs\") pod \"maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:42.532390 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.532324    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64k7n\" (UniqueName: \"kubernetes.io/projected/8136baeb-6550-4400-866d-4796aa76ad35-kube-api-access-64k7n\") pod \"maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:42.974978 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.974942    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:42.977552 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:42.977524    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:43.192334 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:43.192308    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:43.566919 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:43.566816    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"]
Nov 03 21:22:43.569576 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:22:43.569539    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8136baeb_6550_4400_866d_4796aa76ad35.slice/crio-08133533d143ee0f2f66af8d2a733cfc61523d2c30d81b96cc7dafecabf60504 WatchSource:0}: Error finding container 08133533d143ee0f2f66af8d2a733cfc61523d2c30d81b96cc7dafecabf60504: Status 404 returned error can't find the container with id 08133533d143ee0f2f66af8d2a733cfc61523d2c30d81b96cc7dafecabf60504
Nov 03 21:22:43.571058 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:43.571039    4993 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 21:22:44.117962 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:44.117929    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj" event={"ID":"8136baeb-6550-4400-866d-4796aa76ad35","Type":"ContainerStarted","Data":"08133533d143ee0f2f66af8d2a733cfc61523d2c30d81b96cc7dafecabf60504"}
Nov 03 21:22:45.124056 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:45.124017    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj" event={"ID":"8136baeb-6550-4400-866d-4796aa76ad35","Type":"ContainerStarted","Data":"3c2f6c1a586946e1cabc441f8a1dfe65b2e303055d5d85dc361131186a4cddd7"}
Nov 03 21:22:45.124443 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:45.124089    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:22:45.247837 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:45.247773    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj" podStartSLOduration=2.346165509 podStartE2EDuration="3.247749758s" totalImagesPullingTime="901.584249ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:22:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:22:45.244175679 +0000 UTC m=+2567.788882111" watchObservedRunningTime="2026-11-03 21:22:45.247749758 +0000 UTC m=+2567.792456130"
Nov 03 21:22:51.134485 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:22:51.134456    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:23:01.669873 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:23:01.669831    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:23:14.618898 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.618861    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"]
Nov 03 21:23:14.619304 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.619257    4993 cpu_manager.go:414] "containerMap: removing container" podUID="96fbe5e3-4665-45ac-bc64-f128c312fb05" containerName="istio-proxy"
Nov 03 21:23:14.619304 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.619268    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96fbe5e3-4665-45ac-bc64-f128c312fb05" containerName="istio-proxy"
Nov 03 21:23:14.622309 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.622293    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.660355 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.660319    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-model-cache\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.660355 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.660353    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-home\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.660581 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.660376    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-dshm\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.660581 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.660505    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de023273-f049-4434-b0a6-e956a0b296eb-tls-certs\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.660581 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.660550    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jbw7\" (UniqueName: \"kubernetes.io/projected/de023273-f049-4434-b0a6-e956a0b296eb-kube-api-access-7jbw7\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.660774 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.660666    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-tmp-dir\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.690794 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.690768    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-96c0ab\"/\"kube-root-ca.crt\""
Nov 03 21:23:14.690932 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.690849    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-96c0ab\"/\"openshift-service-ca.crt\""
Nov 03 21:23:14.724179 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.724146    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-96c0ab\"/\"default-dockercfg-mbl2x\""
Nov 03 21:23:14.724298 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.724185    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-96c0ab\"/\"simulator-single-main-96c0ab-kserve-self-signed-certs\""
Nov 03 21:23:14.725753 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.725734    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"]
Nov 03 21:23:14.761864 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.761831    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-model-cache\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.762045 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.761878    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-home\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.762045 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.761916    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-dshm\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.762045 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.761951    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de023273-f049-4434-b0a6-e956a0b296eb-tls-certs\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.762045 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.762028    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jbw7\" (UniqueName: \"kubernetes.io/projected/de023273-f049-4434-b0a6-e956a0b296eb-kube-api-access-7jbw7\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.762269 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.762099    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-tmp-dir\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.762333 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.762308    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-model-cache\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.762390 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.762369    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-tmp-dir\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.763789 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.762950    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-home\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.766389 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.766360    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de023273-f049-4434-b0a6-e956a0b296eb-tls-certs\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.769173 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.769150    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-dshm\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.838361 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.838330    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jbw7\" (UniqueName: \"kubernetes.io/projected/de023273-f049-4434-b0a6-e956a0b296eb-kube-api-access-7jbw7\") pod \"simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") " pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:14.942770 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:14.942686    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:15.327734 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:15.327697    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"]
Nov 03 21:23:15.330696 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:23:15.330663    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde023273_f049_4434_b0a6_e956a0b296eb.slice/crio-d2d6aeee71a213d21311d7ca23646c8af9e1ffed696084190db942e3757b142c WatchSource:0}: Error finding container d2d6aeee71a213d21311d7ca23646c8af9e1ffed696084190db942e3757b142c: Status 404 returned error can't find the container with id d2d6aeee71a213d21311d7ca23646c8af9e1ffed696084190db942e3757b142c
Nov 03 21:23:16.265459 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:16.265417    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6" event={"ID":"de023273-f049-4434-b0a6-e956a0b296eb","Type":"ContainerStarted","Data":"9ea9b3eb2e26ddaaddaba5f1b7702e02cbcd569e994491349e42b2cbcd204de0"}
Nov 03 21:23:16.265459 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:16.265462    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6" event={"ID":"de023273-f049-4434-b0a6-e956a0b296eb","Type":"ContainerStarted","Data":"d2d6aeee71a213d21311d7ca23646c8af9e1ffed696084190db942e3757b142c"}
Nov 03 21:23:16.383092 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:16.383030    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6" podStartSLOduration=2.383015902 podStartE2EDuration="2.383015902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:23:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:23:16.381385524 +0000 UTC m=+2598.926091921" watchObservedRunningTime="2026-11-03 21:23:16.383015902 +0000 UTC m=+2598.927722271"
Nov 03 21:23:16.943790 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:16.943750    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:24.943325 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:24.943289    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:24.955882 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:24.955855    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:25.316113 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:23:25.316086    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:23:31.670467 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:23:31.670428    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:24:01.669948 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:24:01.669862    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:24:17.902198 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:17.902159    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"]
Nov 03 21:24:28.334968 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:28.334931    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"]
Nov 03 21:24:28.335406 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:28.335139    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj" podUID="8136baeb-6550-4400-866d-4796aa76ad35" containerName="maas-api" containerID="cri-o://3c2f6c1a586946e1cabc441f8a1dfe65b2e303055d5d85dc361131186a4cddd7" gracePeriod=30
Nov 03 21:24:29.571407 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.571383    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:24:29.592058 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.592029    4993 generic.go:356] "Generic (PLEG): container finished" podID="8136baeb-6550-4400-866d-4796aa76ad35" containerID="3c2f6c1a586946e1cabc441f8a1dfe65b2e303055d5d85dc361131186a4cddd7" exitCode=0
Nov 03 21:24:29.592165 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.592068    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"
Nov 03 21:24:29.592165 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.592105    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj" event={"ID":"8136baeb-6550-4400-866d-4796aa76ad35","Type":"ContainerDied","Data":"3c2f6c1a586946e1cabc441f8a1dfe65b2e303055d5d85dc361131186a4cddd7"}
Nov 03 21:24:29.592165 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.592156    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj" event={"ID":"8136baeb-6550-4400-866d-4796aa76ad35","Type":"ContainerDied","Data":"08133533d143ee0f2f66af8d2a733cfc61523d2c30d81b96cc7dafecabf60504"}
Nov 03 21:24:29.592281 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.592178    4993 scope.go:122] "RemoveContainer" containerID="3c2f6c1a586946e1cabc441f8a1dfe65b2e303055d5d85dc361131186a4cddd7"
Nov 03 21:24:29.600602 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.600570    4993 scope.go:122] "RemoveContainer" containerID="3c2f6c1a586946e1cabc441f8a1dfe65b2e303055d5d85dc361131186a4cddd7"
Nov 03 21:24:29.600920 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:24:29.600898    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c2f6c1a586946e1cabc441f8a1dfe65b2e303055d5d85dc361131186a4cddd7\": container with ID starting with 3c2f6c1a586946e1cabc441f8a1dfe65b2e303055d5d85dc361131186a4cddd7 not found: ID does not exist" containerID="3c2f6c1a586946e1cabc441f8a1dfe65b2e303055d5d85dc361131186a4cddd7"
Nov 03 21:24:29.600978 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.600933    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c2f6c1a586946e1cabc441f8a1dfe65b2e303055d5d85dc361131186a4cddd7"} err="failed to get container status \"3c2f6c1a586946e1cabc441f8a1dfe65b2e303055d5d85dc361131186a4cddd7\": rpc error: code = NotFound desc = could not find container \"3c2f6c1a586946e1cabc441f8a1dfe65b2e303055d5d85dc361131186a4cddd7\": container with ID starting with 3c2f6c1a586946e1cabc441f8a1dfe65b2e303055d5d85dc361131186a4cddd7 not found: ID does not exist"
Nov 03 21:24:29.707621 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.707551    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-maas-api-tls\") pod \"8136baeb-6550-4400-866d-4796aa76ad35\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") "
Nov 03 21:24:29.707621 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.707603    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8136baeb-6550-4400-866d-4796aa76ad35-kube-api-access-64k7n\" (UniqueName: \"kubernetes.io/projected/8136baeb-6550-4400-866d-4796aa76ad35-kube-api-access-64k7n\") pod \"8136baeb-6550-4400-866d-4796aa76ad35\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") "
Nov 03 21:24:29.707826 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.707722    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-metrics-certs\") pod \"8136baeb-6550-4400-866d-4796aa76ad35\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") "
Nov 03 21:24:29.707826 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.707755    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8136baeb-6550-4400-866d-4796aa76ad35-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8136baeb-6550-4400-866d-4796aa76ad35-openshift-service-ca\") pod \"8136baeb-6550-4400-866d-4796aa76ad35\" (UID: \"8136baeb-6550-4400-866d-4796aa76ad35\") "
Nov 03 21:24:29.709729 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.708600    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8136baeb-6550-4400-866d-4796aa76ad35-openshift-service-ca" pod "8136baeb-6550-4400-866d-4796aa76ad35" (UID: "8136baeb-6550-4400-866d-4796aa76ad35"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:24:29.718100 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.718060    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-metrics-certs" pod "8136baeb-6550-4400-866d-4796aa76ad35" (UID: "8136baeb-6550-4400-866d-4796aa76ad35"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:24:29.718216 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.718142    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8136baeb-6550-4400-866d-4796aa76ad35-kube-api-access-64k7n" pod "8136baeb-6550-4400-866d-4796aa76ad35" (UID: "8136baeb-6550-4400-866d-4796aa76ad35"). InnerVolumeSpecName "kube-api-access-64k7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:24:29.718435 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.718392    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-maas-api-tls" pod "8136baeb-6550-4400-866d-4796aa76ad35" (UID: "8136baeb-6550-4400-866d-4796aa76ad35"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:24:29.809033 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.808997    4993 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-maas-api-tls\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:24:29.809033 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.809029    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64k7n\" (UniqueName: \"kubernetes.io/projected/8136baeb-6550-4400-866d-4796aa76ad35-kube-api-access-64k7n\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:24:29.809227 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.809045    4993 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8136baeb-6550-4400-866d-4796aa76ad35-metrics-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:24:29.809227 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:29.809057    4993 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8136baeb-6550-4400-866d-4796aa76ad35-openshift-service-ca\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:24:30.012535 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:30.012509    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"]
Nov 03 21:24:30.020500 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:30.020477    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-96c0ab-84c8d8ddb5-w97pj"]
Nov 03 21:24:31.575723 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:31.575693    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8136baeb-6550-4400-866d-4796aa76ad35" path="/var/lib/kubelet/pods/8136baeb-6550-4400-866d-4796aa76ad35/volumes"
Nov 03 21:24:31.669571 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:24:31.669535    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:24:32.923068 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:32.923025    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6" podUID="de023273-f049-4434-b0a6-e956a0b296eb" containerName="main" containerID="cri-o://9ea9b3eb2e26ddaaddaba5f1b7702e02cbcd569e994491349e42b2cbcd204de0" gracePeriod=45
Nov 03 21:24:33.184860 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.184806    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:24:33.239206 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.239182    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-dshm\") pod \"de023273-f049-4434-b0a6-e956a0b296eb\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") "
Nov 03 21:24:33.239322 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.239256    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-home\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-home\") pod \"de023273-f049-4434-b0a6-e956a0b296eb\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") "
Nov 03 21:24:33.239322 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.239276    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-model-cache\") pod \"de023273-f049-4434-b0a6-e956a0b296eb\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") "
Nov 03 21:24:33.239446 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.239346    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de023273-f049-4434-b0a6-e956a0b296eb-kube-api-access-7jbw7\" (UniqueName: \"kubernetes.io/projected/de023273-f049-4434-b0a6-e956a0b296eb-kube-api-access-7jbw7\") pod \"de023273-f049-4434-b0a6-e956a0b296eb\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") "
Nov 03 21:24:33.239446 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.239381    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-tmp-dir\") pod \"de023273-f049-4434-b0a6-e956a0b296eb\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") "
Nov 03 21:24:33.239446 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.239409    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/de023273-f049-4434-b0a6-e956a0b296eb-tls-certs\" (UniqueName: \"kubernetes.io/secret/de023273-f049-4434-b0a6-e956a0b296eb-tls-certs\") pod \"de023273-f049-4434-b0a6-e956a0b296eb\" (UID: \"de023273-f049-4434-b0a6-e956a0b296eb\") "
Nov 03 21:24:33.239613 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.239560    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-home" pod "de023273-f049-4434-b0a6-e956a0b296eb" (UID: "de023273-f049-4434-b0a6-e956a0b296eb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:24:33.239613 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.239572    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-model-cache" pod "de023273-f049-4434-b0a6-e956a0b296eb" (UID: "de023273-f049-4434-b0a6-e956a0b296eb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:24:33.239742 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.239691    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-tmp-dir" pod "de023273-f049-4434-b0a6-e956a0b296eb" (UID: "de023273-f049-4434-b0a6-e956a0b296eb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:24:33.239742 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.239733    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:24:33.239809 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.239754    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:24:33.241507 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.241481    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-dshm" pod "de023273-f049-4434-b0a6-e956a0b296eb" (UID: "de023273-f049-4434-b0a6-e956a0b296eb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:24:33.241675 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.241641    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de023273-f049-4434-b0a6-e956a0b296eb-tls-certs" pod "de023273-f049-4434-b0a6-e956a0b296eb" (UID: "de023273-f049-4434-b0a6-e956a0b296eb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:24:33.241675 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.241664    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de023273-f049-4434-b0a6-e956a0b296eb-kube-api-access-7jbw7" pod "de023273-f049-4434-b0a6-e956a0b296eb" (UID: "de023273-f049-4434-b0a6-e956a0b296eb"). InnerVolumeSpecName "kube-api-access-7jbw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:24:33.340365 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.340337    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:24:33.340365 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.340365    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de023273-f049-4434-b0a6-e956a0b296eb-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:24:33.340365 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.340378    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de023273-f049-4434-b0a6-e956a0b296eb-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:24:33.340613 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.340393    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jbw7\" (UniqueName: \"kubernetes.io/projected/de023273-f049-4434-b0a6-e956a0b296eb-kube-api-access-7jbw7\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:24:33.615920 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.615891    4993 generic.go:356] "Generic (PLEG): container finished" podID="de023273-f049-4434-b0a6-e956a0b296eb" containerID="9ea9b3eb2e26ddaaddaba5f1b7702e02cbcd569e994491349e42b2cbcd204de0" exitCode=0
Nov 03 21:24:33.616095 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.615937    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"
Nov 03 21:24:33.616095 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.615942    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6" event={"ID":"de023273-f049-4434-b0a6-e956a0b296eb","Type":"ContainerDied","Data":"9ea9b3eb2e26ddaaddaba5f1b7702e02cbcd569e994491349e42b2cbcd204de0"}
Nov 03 21:24:33.616095 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.616080    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6" event={"ID":"de023273-f049-4434-b0a6-e956a0b296eb","Type":"ContainerDied","Data":"d2d6aeee71a213d21311d7ca23646c8af9e1ffed696084190db942e3757b142c"}
Nov 03 21:24:33.616271 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.616111    4993 scope.go:122] "RemoveContainer" containerID="9ea9b3eb2e26ddaaddaba5f1b7702e02cbcd569e994491349e42b2cbcd204de0"
Nov 03 21:24:33.624537 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.624519    4993 scope.go:122] "RemoveContainer" containerID="9ea9b3eb2e26ddaaddaba5f1b7702e02cbcd569e994491349e42b2cbcd204de0"
Nov 03 21:24:33.624840 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:24:33.624817    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ea9b3eb2e26ddaaddaba5f1b7702e02cbcd569e994491349e42b2cbcd204de0\": container with ID starting with 9ea9b3eb2e26ddaaddaba5f1b7702e02cbcd569e994491349e42b2cbcd204de0 not found: ID does not exist" containerID="9ea9b3eb2e26ddaaddaba5f1b7702e02cbcd569e994491349e42b2cbcd204de0"
Nov 03 21:24:33.624944 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.624845    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ea9b3eb2e26ddaaddaba5f1b7702e02cbcd569e994491349e42b2cbcd204de0"} err="failed to get container status \"9ea9b3eb2e26ddaaddaba5f1b7702e02cbcd569e994491349e42b2cbcd204de0\": rpc error: code = NotFound desc = could not find container \"9ea9b3eb2e26ddaaddaba5f1b7702e02cbcd569e994491349e42b2cbcd204de0\": container with ID starting with 9ea9b3eb2e26ddaaddaba5f1b7702e02cbcd569e994491349e42b2cbcd204de0 not found: ID does not exist"
Nov 03 21:24:33.778825 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.778789    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"]
Nov 03 21:24:33.782666 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:33.782638    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-96c0ab/simulator-single-main-96c0ab-kserve-7fdf696db5-27wr6"]
Nov 03 21:24:35.576549 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:24:35.576514    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de023273-f049-4434-b0a6-e956a0b296eb" path="/var/lib/kubelet/pods/de023273-f049-4434-b0a6-e956a0b296eb/volumes"
Nov 03 21:25:01.670443 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:25:01.670403    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:25:13.958784 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:13.958703    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"]
Nov 03 21:25:13.959293 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:13.958906    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg" podUID="2ff49561-6d05-43e2-9a27-3fa48f5be683" containerName="manager" containerID="cri-o://2c138666cee6fcfb86cc20edc43ef2808aaf35809115341c8e4b3eebc65d30fe" gracePeriod=10
Nov 03 21:25:14.203451 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:14.203428    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"
Nov 03 21:25:14.307513 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:14.307488    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ff49561-6d05-43e2-9a27-3fa48f5be683-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ff49561-6d05-43e2-9a27-3fa48f5be683-extensions-socket-volume\") pod \"2ff49561-6d05-43e2-9a27-3fa48f5be683\" (UID: \"2ff49561-6d05-43e2-9a27-3fa48f5be683\") "
Nov 03 21:25:14.307706 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:14.307598    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ff49561-6d05-43e2-9a27-3fa48f5be683-kube-api-access-kzmhp\" (UniqueName: \"kubernetes.io/projected/2ff49561-6d05-43e2-9a27-3fa48f5be683-kube-api-access-kzmhp\") pod \"2ff49561-6d05-43e2-9a27-3fa48f5be683\" (UID: \"2ff49561-6d05-43e2-9a27-3fa48f5be683\") "
Nov 03 21:25:14.307954 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:14.307924    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ff49561-6d05-43e2-9a27-3fa48f5be683-extensions-socket-volume" pod "2ff49561-6d05-43e2-9a27-3fa48f5be683" (UID: "2ff49561-6d05-43e2-9a27-3fa48f5be683"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:25:14.309813 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:14.309795    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ff49561-6d05-43e2-9a27-3fa48f5be683-kube-api-access-kzmhp" pod "2ff49561-6d05-43e2-9a27-3fa48f5be683" (UID: "2ff49561-6d05-43e2-9a27-3fa48f5be683"). InnerVolumeSpecName "kube-api-access-kzmhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:25:14.409025 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:14.408983    4993 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ff49561-6d05-43e2-9a27-3fa48f5be683-extensions-socket-volume\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:25:14.409025 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:14.409023    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzmhp\" (UniqueName: \"kubernetes.io/projected/2ff49561-6d05-43e2-9a27-3fa48f5be683-kube-api-access-kzmhp\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:25:14.822731 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:14.822699    4993 generic.go:356] "Generic (PLEG): container finished" podID="2ff49561-6d05-43e2-9a27-3fa48f5be683" containerID="2c138666cee6fcfb86cc20edc43ef2808aaf35809115341c8e4b3eebc65d30fe" exitCode=0
Nov 03 21:25:14.822915 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:14.822744    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"
Nov 03 21:25:14.822915 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:14.822753    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg" event={"ID":"2ff49561-6d05-43e2-9a27-3fa48f5be683","Type":"ContainerDied","Data":"2c138666cee6fcfb86cc20edc43ef2808aaf35809115341c8e4b3eebc65d30fe"}
Nov 03 21:25:14.822915 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:14.822794    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg" event={"ID":"2ff49561-6d05-43e2-9a27-3fa48f5be683","Type":"ContainerDied","Data":"f0734f3ed521553e9e3c44b9b086fa6a91ec4f27edc9ca1e6c1e2718a643e1f0"}
Nov 03 21:25:14.822915 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:14.822815    4993 scope.go:122] "RemoveContainer" containerID="2c138666cee6fcfb86cc20edc43ef2808aaf35809115341c8e4b3eebc65d30fe"
Nov 03 21:25:14.835507 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:14.835487    4993 scope.go:122] "RemoveContainer" containerID="2c138666cee6fcfb86cc20edc43ef2808aaf35809115341c8e4b3eebc65d30fe"
Nov 03 21:25:14.835838 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:25:14.835821    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c138666cee6fcfb86cc20edc43ef2808aaf35809115341c8e4b3eebc65d30fe\": container with ID starting with 2c138666cee6fcfb86cc20edc43ef2808aaf35809115341c8e4b3eebc65d30fe not found: ID does not exist" containerID="2c138666cee6fcfb86cc20edc43ef2808aaf35809115341c8e4b3eebc65d30fe"
Nov 03 21:25:14.835897 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:14.835848    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c138666cee6fcfb86cc20edc43ef2808aaf35809115341c8e4b3eebc65d30fe"} err="failed to get container status \"2c138666cee6fcfb86cc20edc43ef2808aaf35809115341c8e4b3eebc65d30fe\": rpc error: code = NotFound desc = could not find container \"2c138666cee6fcfb86cc20edc43ef2808aaf35809115341c8e4b3eebc65d30fe\": container with ID starting with 2c138666cee6fcfb86cc20edc43ef2808aaf35809115341c8e4b3eebc65d30fe not found: ID does not exist"
Nov 03 21:25:14.986209 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:14.986184    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"]
Nov 03 21:25:15.036227 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:15.036190    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-f9dzg"]
Nov 03 21:25:15.575766 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:25:15.575733    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ff49561-6d05-43e2-9a27-3fa48f5be683" path="/var/lib/kubelet/pods/2ff49561-6d05-43e2-9a27-3fa48f5be683/volumes"
Nov 03 21:25:31.668692 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:25:31.668641    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:26:01.668727 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:26:01.668693    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:26:31.669182 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:26:31.669146    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:27:01.668778 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:27:01.668740    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:27:03.974797 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:03.974752    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:27:03.980848 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:03.975363    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg" podUID="ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:27:04.120792 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:04.120749    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:27:04.121120 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:04.121074    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7" podUID="b056b057-0700-4450-b07d-f19b53d5584a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:27:25.283251 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.283219    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"]
Nov 03 21:27:25.283703 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.283686    4993 cpu_manager.go:414] "containerMap: removing container" podUID="de023273-f049-4434-b0a6-e956a0b296eb" containerName="main"
Nov 03 21:27:25.283754 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.283703    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de023273-f049-4434-b0a6-e956a0b296eb" containerName="main"
Nov 03 21:27:25.283754 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.283737    4993 cpu_manager.go:414] "containerMap: removing container" podUID="2ff49561-6d05-43e2-9a27-3fa48f5be683" containerName="manager"
Nov 03 21:27:25.283754 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.283744    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ff49561-6d05-43e2-9a27-3fa48f5be683" containerName="manager"
Nov 03 21:27:25.283754 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.283754    4993 cpu_manager.go:414] "containerMap: removing container" podUID="8136baeb-6550-4400-866d-4796aa76ad35" containerName="maas-api"
Nov 03 21:27:25.283883 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.283760    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8136baeb-6550-4400-866d-4796aa76ad35" containerName="maas-api"
Nov 03 21:27:25.283883 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.283823    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="8136baeb-6550-4400-866d-4796aa76ad35" containerName="maas-api"
Nov 03 21:27:25.283883 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.283833    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="de023273-f049-4434-b0a6-e956a0b296eb" containerName="main"
Nov 03 21:27:25.283883 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.283842    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ff49561-6d05-43e2-9a27-3fa48f5be683" containerName="manager"
Nov 03 21:27:25.286822 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.286805    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.339428 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.339393    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-single-main-eb1249-openshift-default-dockercfg-wh4lf\""
Nov 03 21:27:25.347266 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.347236    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"]
Nov 03 21:27:25.402232 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.402191    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-credential-socket\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.402232 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.402224    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-podinfo\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.402473 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.402250    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqfl8\" (UniqueName: \"kubernetes.io/projected/7450de41-5cbb-4aeb-8141-b71a01ca7189-kube-api-access-kqfl8\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.402473 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.402350    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-workload-socket\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.402473 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.402402    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-workload-certs\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.402473 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.402459    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-token\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.402726 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.402484    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-envoy\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.402726 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.402589    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-data\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.402726 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.402642    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7450de41-5cbb-4aeb-8141-b71a01ca7189-istiod-ca-cert\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.503413 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.503375    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-workload-socket\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.503609 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.503433    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-workload-certs\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.503609 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.503500    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-token\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.503609 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.503555    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-envoy\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.503609 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.503593    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-data\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.503870 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.503658    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7450de41-5cbb-4aeb-8141-b71a01ca7189-istiod-ca-cert\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.503870 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.503683    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-credential-socket\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.503870 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.503711    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-podinfo\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.503870 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.503747    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqfl8\" (UniqueName: \"kubernetes.io/projected/7450de41-5cbb-4aeb-8141-b71a01ca7189-kube-api-access-kqfl8\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.503870 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.503819    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-workload-socket\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.504096 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.503882    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-workload-certs\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.504096 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.503998    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-data\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.504096 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.504080    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-credential-socket\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.504460 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.504432    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7450de41-5cbb-4aeb-8141-b71a01ca7189-istiod-ca-cert\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.505940 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.505916    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-envoy\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.506180 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.506164    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-podinfo\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.587691 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.587605    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-token\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.587832 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.587707    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqfl8\" (UniqueName: \"kubernetes.io/projected/7450de41-5cbb-4aeb-8141-b71a01ca7189-kube-api-access-kqfl8\") pod \"e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") " pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:25.600220 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:25.600200    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:26.025238 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:26.025215    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"]
Nov 03 21:27:26.027736 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:27:26.027705    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7450de41_5cbb_4aeb_8141_b71a01ca7189.slice/crio-e2634ce9adaab13b592613c6258903da19e72fcce5ab062cbeb1504bc6816bc2 WatchSource:0}: Error finding container e2634ce9adaab13b592613c6258903da19e72fcce5ab062cbeb1504bc6816bc2: Status 404 returned error can't find the container with id e2634ce9adaab13b592613c6258903da19e72fcce5ab062cbeb1504bc6816bc2
Nov 03 21:27:26.029980 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:26.029946    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:27:26.030080 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:26.030021    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:27:26.030080 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:26.030070    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:27:26.433717 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:26.433611    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6" event={"ID":"7450de41-5cbb-4aeb-8141-b71a01ca7189","Type":"ContainerStarted","Data":"c5dc9f89f6434c709fa075ab0f2609ce174bd3de034f7bdb86de0b25c32da5ae"}
Nov 03 21:27:26.433717 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:26.433686    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6" event={"ID":"7450de41-5cbb-4aeb-8141-b71a01ca7189","Type":"ContainerStarted","Data":"e2634ce9adaab13b592613c6258903da19e72fcce5ab062cbeb1504bc6816bc2"}
Nov 03 21:27:26.555764 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:26.555708    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6" podStartSLOduration=1.5556948099999999 podStartE2EDuration="1.55569481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:27:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:27:26.554202395 +0000 UTC m=+2849.098908763" watchObservedRunningTime="2026-11-03 21:27:26.55569481 +0000 UTC m=+2849.100401179"
Nov 03 21:27:26.600957 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:26.600933    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:27.606135 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:27.606107    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:28.441585 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:28.441553    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:28.442501 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:28.442482    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:31.669333 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:27:31.669295    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:27:33.287918 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:33.287885    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"]
Nov 03 21:27:33.291680 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:33.291648    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"
Nov 03 21:27:33.372232 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:33.372205    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmkjt\" (UniqueName: \"kubernetes.io/projected/f2f7f5a4-95df-49d6-95aa-04062fbde687-kube-api-access-tmkjt\") pod \"payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r\" (UID: \"f2f7f5a4-95df-49d6-95aa-04062fbde687\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"
Nov 03 21:27:33.372386 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:33.372253    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2f7f5a4-95df-49d6-95aa-04062fbde687-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r\" (UID: \"f2f7f5a4-95df-49d6-95aa-04062fbde687\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"
Nov 03 21:27:33.393533 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:33.393512    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-single-m-f4dcec6e-dockercfg-lxt4r\""
Nov 03 21:27:33.393742 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:33.393724    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-single-main-eb1249\""
Nov 03 21:27:33.395082 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:33.395064    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"]
Nov 03 21:27:33.473228 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:33.473195    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmkjt\" (UniqueName: \"kubernetes.io/projected/f2f7f5a4-95df-49d6-95aa-04062fbde687-kube-api-access-tmkjt\") pod \"payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r\" (UID: \"f2f7f5a4-95df-49d6-95aa-04062fbde687\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"
Nov 03 21:27:33.473406 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:33.473258    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2f7f5a4-95df-49d6-95aa-04062fbde687-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r\" (UID: \"f2f7f5a4-95df-49d6-95aa-04062fbde687\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"
Nov 03 21:27:33.473978 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:33.473953    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2f7f5a4-95df-49d6-95aa-04062fbde687-plugins-config-volume\") pod \"payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r\" (UID: \"f2f7f5a4-95df-49d6-95aa-04062fbde687\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"
Nov 03 21:27:33.535168 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:33.535136    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmkjt\" (UniqueName: \"kubernetes.io/projected/f2f7f5a4-95df-49d6-95aa-04062fbde687-kube-api-access-tmkjt\") pod \"payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r\" (UID: \"f2f7f5a4-95df-49d6-95aa-04062fbde687\") " pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"
Nov 03 21:27:33.606610 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:33.606547    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"
Nov 03 21:27:33.976258 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:33.976230    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"]
Nov 03 21:27:33.978272 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:27:33.978234    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2f7f5a4_95df_49d6_95aa_04062fbde687.slice/crio-002496d51a8b5fa8aeb103e86b44a9130cc1a784a9044b33a7b04e782c9cc48b WatchSource:0}: Error finding container 002496d51a8b5fa8aeb103e86b44a9130cc1a784a9044b33a7b04e782c9cc48b: Status 404 returned error can't find the container with id 002496d51a8b5fa8aeb103e86b44a9130cc1a784a9044b33a7b04e782c9cc48b
Nov 03 21:27:34.469790 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:34.469756    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r" event={"ID":"f2f7f5a4-95df-49d6-95aa-04062fbde687","Type":"ContainerStarted","Data":"002496d51a8b5fa8aeb103e86b44a9130cc1a784a9044b33a7b04e782c9cc48b"}
Nov 03 21:27:35.470141 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:35.470105    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"]
Nov 03 21:27:35.474780 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:35.473979    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:35.526384 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:35.526352    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-eb1249-serving-cert\""
Nov 03 21:27:35.533578 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:35.533548    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"]
Nov 03 21:27:35.606313 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:35.606276    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx9ls\" (UniqueName: \"kubernetes.io/projected/8551d21b-2481-4570-a9dd-57a3a01ba9e7-kube-api-access-gx9ls\") pod \"maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:35.606487 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:35.606332    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:35.606487 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:35.606370    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-metrics-certs\") pod \"maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:35.606487 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:35.606401    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8551d21b-2481-4570-a9dd-57a3a01ba9e7-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:35.708412 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:35.707497    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx9ls\" (UniqueName: \"kubernetes.io/projected/8551d21b-2481-4570-a9dd-57a3a01ba9e7-kube-api-access-gx9ls\") pod \"maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:35.708412 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:35.707577    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:35.708412 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:35.707604    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-metrics-certs\") pod \"maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:35.708412 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:35.707650    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8551d21b-2481-4570-a9dd-57a3a01ba9e7-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:35.708412 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:27:35.707732    4993 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-serving-cert: secret "maas-api-e2e-worker-single-main-eb1249-serving-cert" not found
Nov 03 21:27:35.708412 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:27:35.707793    4993 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-maas-api-tls podName:8551d21b-2481-4570-a9dd-57a3a01ba9e7 nodeName:}" failed. No retries permitted until 2026-11-03 21:27:36.207774158 +0000 UTC m=+2858.752480545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-maas-api-tls") pod "maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2" (UID: "8551d21b-2481-4570-a9dd-57a3a01ba9e7") : secret "maas-api-e2e-worker-single-main-eb1249-serving-cert" not found
Nov 03 21:27:35.708412 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:35.708372    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8551d21b-2481-4570-a9dd-57a3a01ba9e7-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:35.710560 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:35.710530    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-metrics-certs\") pod \"maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:35.768906 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:35.768798    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx9ls\" (UniqueName: \"kubernetes.io/projected/8551d21b-2481-4570-a9dd-57a3a01ba9e7-kube-api-access-gx9ls\") pod \"maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:36.212843 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:36.212804    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:36.215585 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:36.215561    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:36.389715 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:36.389682    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:36.772557 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:36.772408    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"]
Nov 03 21:27:36.775154 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:27:36.775124    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8551d21b_2481_4570_a9dd_57a3a01ba9e7.slice/crio-19a67ea7ea805b7834706d2e0d98d1bac5b5643750267f519fac05ea18189043 WatchSource:0}: Error finding container 19a67ea7ea805b7834706d2e0d98d1bac5b5643750267f519fac05ea18189043: Status 404 returned error can't find the container with id 19a67ea7ea805b7834706d2e0d98d1bac5b5643750267f519fac05ea18189043
Nov 03 21:27:37.487483 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:37.487441    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r" event={"ID":"f2f7f5a4-95df-49d6-95aa-04062fbde687","Type":"ContainerStarted","Data":"4b83ea32dec7c071ba90aae9e7cd1b42b5d561b128d566dc7db25dd970e29762"}
Nov 03 21:27:37.487711 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:37.487577    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"
Nov 03 21:27:37.488933 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:37.488906    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2" event={"ID":"8551d21b-2481-4570-a9dd-57a3a01ba9e7","Type":"ContainerStarted","Data":"19a67ea7ea805b7834706d2e0d98d1bac5b5643750267f519fac05ea18189043"}
Nov 03 21:27:37.604231 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:37.604179    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r" podStartSLOduration=1.964517955 podStartE2EDuration="4.604166641s" totalImagesPullingTime="2.639648686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:27:37.60226305 +0000 UTC m=+2860.146969417" watchObservedRunningTime="2026-11-03 21:27:37.604166641 +0000 UTC m=+2860.148873019"
Nov 03 21:27:38.496158 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:38.496119    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2" event={"ID":"8551d21b-2481-4570-a9dd-57a3a01ba9e7","Type":"ContainerStarted","Data":"51f6baf2ad32ffc6a5ac0ede5c5efc475ec97d89e2883b1323de29408817ea80"}
Nov 03 21:27:38.615788 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:38.615726    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2" podStartSLOduration=2.814977519 podStartE2EDuration="3.615711624s" totalImagesPullingTime="800.734105ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:27:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:27:38.613982844 +0000 UTC m=+2861.158689215" watchObservedRunningTime="2026-11-03 21:27:38.615711624 +0000 UTC m=+2861.160417972"
Nov 03 21:27:39.501687 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:39.501652    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:45.511158 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:45.511132    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:27:48.500024 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:48.499995    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"
Nov 03 21:27:53.566760 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:53.566728    4993 generic.go:356] "Generic (PLEG): container finished" podID="7450de41-5cbb-4aeb-8141-b71a01ca7189" containerID="c5dc9f89f6434c709fa075ab0f2609ce174bd3de034f7bdb86de0b25c32da5ae" exitCode=0
Nov 03 21:27:53.566760 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:53.566765    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6" event={"ID":"7450de41-5cbb-4aeb-8141-b71a01ca7189","Type":"ContainerDied","Data":"c5dc9f89f6434c709fa075ab0f2609ce174bd3de034f7bdb86de0b25c32da5ae"}
Nov 03 21:27:53.567212 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:53.567137    4993 scope.go:122] "RemoveContainer" containerID="c5dc9f89f6434c709fa075ab0f2609ce174bd3de034f7bdb86de0b25c32da5ae"
Nov 03 21:27:53.568288 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:53.568262    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:27:53.568356 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:53.568336    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:27:53.568397 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:53.568384    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:27:54.592537 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:54.592502    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6" event={"ID":"7450de41-5cbb-4aeb-8141-b71a01ca7189","Type":"ContainerStarted","Data":"59aa240593d783d294d1b2d905f95abda8e841ddef59aaa2eb39513564700c14"}
Nov 03 21:27:54.601021 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:54.601004    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:54.602204 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:54.602183    4993 patch_prober.go:29] interesting pod/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 03 21:27:54.602351 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:54.602228    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6" podUID="7450de41-5cbb-4aeb-8141-b71a01ca7189" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 03 21:27:55.601193 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:55.601159    4993 patch_prober.go:29] interesting pod/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 03 21:27:55.601597 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:55.601212    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6" podUID="7450de41-5cbb-4aeb-8141-b71a01ca7189" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 03 21:27:56.600517 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:56.600483    4993 patch_prober.go:29] interesting pod/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 03 21:27:56.600747 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:56.600551    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6" podUID="7450de41-5cbb-4aeb-8141-b71a01ca7189" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 03 21:27:57.601511 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:57.601472    4993 patch_prober.go:29] interesting pod/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 03 21:27:57.602007 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:57.601528    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6" podUID="7450de41-5cbb-4aeb-8141-b71a01ca7189" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 03 21:27:57.982872 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:57.982839    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"]
Nov 03 21:27:57.988440 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:57.988407    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.017920 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.017891    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-model-cache\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.018034 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.018016    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-home\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.018085 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.018077    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55e7133b-3b3d-4c59-a06e-d18883b6dca5-tls-certs\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.018122 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.018099    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-tmp-dir\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.018122 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.018113    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-dshm\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.018341 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.018316    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6wfg\" (UniqueName: \"kubernetes.io/projected/55e7133b-3b3d-4c59-a06e-d18883b6dca5-kube-api-access-w6wfg\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.040805 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.040779    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"]
Nov 03 21:27:58.088019 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.087921    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-eb1249\"/\"openshift-service-ca.crt\""
Nov 03 21:27:58.088019 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.087942    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-eb1249\"/\"default-dockercfg-4g8fz\""
Nov 03 21:27:58.088019 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.087994    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-eb1249\"/\"kube-root-ca.crt\""
Nov 03 21:27:58.090039 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.090017    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-eb1249\"/\"simulator-single-main-eb1249-kserve-self-signed-certs\""
Nov 03 21:27:58.119716 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.119688    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-home\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.119883 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.119758    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55e7133b-3b3d-4c59-a06e-d18883b6dca5-tls-certs\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.119883 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.119794    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-tmp-dir\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.119883 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.119815    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-dshm\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.120062 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.119890    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6wfg\" (UniqueName: \"kubernetes.io/projected/55e7133b-3b3d-4c59-a06e-d18883b6dca5-kube-api-access-w6wfg\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.120062 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.119935    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-model-cache\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.120394 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.120141    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-home\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.120394 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.120235    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-model-cache\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.120570 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.120528    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-tmp-dir\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.123845 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.123816    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-dshm\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.123845 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.123835    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55e7133b-3b3d-4c59-a06e-d18883b6dca5-tls-certs\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.189651 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.189592    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6wfg\" (UniqueName: \"kubernetes.io/projected/55e7133b-3b3d-4c59-a06e-d18883b6dca5-kube-api-access-w6wfg\") pod \"simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") " pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.308673 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.308570    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:27:58.442529 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.442229    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:27:58.601160 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.601081    4993 patch_prober.go:29] interesting pod/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 03 21:27:58.601160 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.601140    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6" podUID="7450de41-5cbb-4aeb-8141-b71a01ca7189" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 03 21:27:58.703263 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:58.703225    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"]
Nov 03 21:27:58.709667 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:27:58.709623    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55e7133b_3b3d_4c59_a06e_d18883b6dca5.slice/crio-a3c18c3b417160f806fbf4cd7cc5d9191d210538cd290b4857c9914c4c5ebb83 WatchSource:0}: Error finding container a3c18c3b417160f806fbf4cd7cc5d9191d210538cd290b4857c9914c4c5ebb83: Status 404 returned error can't find the container with id a3c18c3b417160f806fbf4cd7cc5d9191d210538cd290b4857c9914c4c5ebb83
Nov 03 21:27:59.601477 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:59.601444    4993 patch_prober.go:29] interesting pod/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Nov 03 21:27:59.601669 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:59.601494    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6" podUID="7450de41-5cbb-4aeb-8141-b71a01ca7189" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Nov 03 21:27:59.615698 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:59.615656    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg" event={"ID":"55e7133b-3b3d-4c59-a06e-d18883b6dca5","Type":"ContainerStarted","Data":"e81cde2d6b05d7915bc0e113728cab7e5814805a98d7d5b7839fab1a216d3844"}
Nov 03 21:27:59.615826 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:59.615708    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg" event={"ID":"55e7133b-3b3d-4c59-a06e-d18883b6dca5","Type":"ContainerStarted","Data":"a3c18c3b417160f806fbf4cd7cc5d9191d210538cd290b4857c9914c4c5ebb83"}
Nov 03 21:27:59.732806 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:27:59.732751    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg" podStartSLOduration=2.732737479 podStartE2EDuration="2.732737479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:27:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:27:59.732306258 +0000 UTC m=+2882.277012629" watchObservedRunningTime="2026-11-03 21:27:59.732737479 +0000 UTC m=+2882.277443849"
Nov 03 21:28:00.309423 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:28:00.309390    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:28:00.604731 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:28:00.604661    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:28:00.605605 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:28:00.605589    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:28:01.668974 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:28:01.668940    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:28:08.309389 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:28:08.309309    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:28:08.322224 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:28:08.322197    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:28:08.665210 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:28:08.665136    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:28:31.670845 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:28:31.670805    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:28:58.696237 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:28:58.696202    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"]
Nov 03 21:29:01.672167 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:29:01.672131    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:29:05.655723 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:05.655689    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"]
Nov 03 21:29:10.656778 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:10.656718    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r" podUID="f2f7f5a4-95df-49d6-95aa-04062fbde687" containerName="payload-processing" containerID="cri-o://4b83ea32dec7c071ba90aae9e7cd1b42b5d561b128d566dc7db25dd970e29762" gracePeriod=25
Nov 03 21:29:10.905217 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:10.905195    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"
Nov 03 21:29:10.925142 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:10.925086    4993 generic.go:356] "Generic (PLEG): container finished" podID="f2f7f5a4-95df-49d6-95aa-04062fbde687" containerID="4b83ea32dec7c071ba90aae9e7cd1b42b5d561b128d566dc7db25dd970e29762" exitCode=0
Nov 03 21:29:10.925142 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:10.925128    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"
Nov 03 21:29:10.925297 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:10.925148    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r" event={"ID":"f2f7f5a4-95df-49d6-95aa-04062fbde687","Type":"ContainerDied","Data":"4b83ea32dec7c071ba90aae9e7cd1b42b5d561b128d566dc7db25dd970e29762"}
Nov 03 21:29:10.925297 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:10.925186    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r" event={"ID":"f2f7f5a4-95df-49d6-95aa-04062fbde687","Type":"ContainerDied","Data":"002496d51a8b5fa8aeb103e86b44a9130cc1a784a9044b33a7b04e782c9cc48b"}
Nov 03 21:29:10.925297 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:10.925208    4993 scope.go:122] "RemoveContainer" containerID="4b83ea32dec7c071ba90aae9e7cd1b42b5d561b128d566dc7db25dd970e29762"
Nov 03 21:29:10.933354 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:10.933338    4993 scope.go:122] "RemoveContainer" containerID="4b83ea32dec7c071ba90aae9e7cd1b42b5d561b128d566dc7db25dd970e29762"
Nov 03 21:29:10.933665 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:29:10.933639    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b83ea32dec7c071ba90aae9e7cd1b42b5d561b128d566dc7db25dd970e29762\": container with ID starting with 4b83ea32dec7c071ba90aae9e7cd1b42b5d561b128d566dc7db25dd970e29762 not found: ID does not exist" containerID="4b83ea32dec7c071ba90aae9e7cd1b42b5d561b128d566dc7db25dd970e29762"
Nov 03 21:29:10.933760 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:10.933672    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b83ea32dec7c071ba90aae9e7cd1b42b5d561b128d566dc7db25dd970e29762"} err="failed to get container status \"4b83ea32dec7c071ba90aae9e7cd1b42b5d561b128d566dc7db25dd970e29762\": rpc error: code = NotFound desc = could not find container \"4b83ea32dec7c071ba90aae9e7cd1b42b5d561b128d566dc7db25dd970e29762\": container with ID starting with 4b83ea32dec7c071ba90aae9e7cd1b42b5d561b128d566dc7db25dd970e29762 not found: ID does not exist"
Nov 03 21:29:11.036168 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:11.036137    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2f7f5a4-95df-49d6-95aa-04062fbde687-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2f7f5a4-95df-49d6-95aa-04062fbde687-plugins-config-volume\") pod \"f2f7f5a4-95df-49d6-95aa-04062fbde687\" (UID: \"f2f7f5a4-95df-49d6-95aa-04062fbde687\") "
Nov 03 21:29:11.036323 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:11.036216    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2f7f5a4-95df-49d6-95aa-04062fbde687-kube-api-access-tmkjt\" (UniqueName: \"kubernetes.io/projected/f2f7f5a4-95df-49d6-95aa-04062fbde687-kube-api-access-tmkjt\") pod \"f2f7f5a4-95df-49d6-95aa-04062fbde687\" (UID: \"f2f7f5a4-95df-49d6-95aa-04062fbde687\") "
Nov 03 21:29:11.036575 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:11.036552    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2f7f5a4-95df-49d6-95aa-04062fbde687-plugins-config-volume" pod "f2f7f5a4-95df-49d6-95aa-04062fbde687" (UID: "f2f7f5a4-95df-49d6-95aa-04062fbde687"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:29:11.038367 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:11.038340    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2f7f5a4-95df-49d6-95aa-04062fbde687-kube-api-access-tmkjt" pod "f2f7f5a4-95df-49d6-95aa-04062fbde687" (UID: "f2f7f5a4-95df-49d6-95aa-04062fbde687"). InnerVolumeSpecName "kube-api-access-tmkjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:29:11.137717 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:11.137683    4993 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2f7f5a4-95df-49d6-95aa-04062fbde687-plugins-config-volume\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:11.137717 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:11.137711    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmkjt\" (UniqueName: \"kubernetes.io/projected/f2f7f5a4-95df-49d6-95aa-04062fbde687-kube-api-access-tmkjt\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:11.343610 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:11.343580    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"]
Nov 03 21:29:11.347435 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:11.347410    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-single-main-eb1249-86dd88c957lb4r"]
Nov 03 21:29:11.575324 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:11.575248    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2f7f5a4-95df-49d6-95aa-04062fbde687" path="/var/lib/kubelet/pods/f2f7f5a4-95df-49d6-95aa-04062fbde687/volumes"
Nov 03 21:29:11.655000 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:11.654963    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"]
Nov 03 21:29:11.655194 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:11.655168    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2" podUID="8551d21b-2481-4570-a9dd-57a3a01ba9e7" containerName="maas-api" containerID="cri-o://51f6baf2ad32ffc6a5ac0ede5c5efc475ec97d89e2883b1323de29408817ea80" gracePeriod=30
Nov 03 21:29:12.902929 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.902905    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:29:12.935267 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.935244    4993 generic.go:356] "Generic (PLEG): container finished" podID="8551d21b-2481-4570-a9dd-57a3a01ba9e7" containerID="51f6baf2ad32ffc6a5ac0ede5c5efc475ec97d89e2883b1323de29408817ea80" exitCode=0
Nov 03 21:29:12.935374 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.935285    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"
Nov 03 21:29:12.935417 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.935289    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2" event={"ID":"8551d21b-2481-4570-a9dd-57a3a01ba9e7","Type":"ContainerDied","Data":"51f6baf2ad32ffc6a5ac0ede5c5efc475ec97d89e2883b1323de29408817ea80"}
Nov 03 21:29:12.935455 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.935444    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2" event={"ID":"8551d21b-2481-4570-a9dd-57a3a01ba9e7","Type":"ContainerDied","Data":"19a67ea7ea805b7834706d2e0d98d1bac5b5643750267f519fac05ea18189043"}
Nov 03 21:29:12.935499 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.935479    4993 scope.go:122] "RemoveContainer" containerID="51f6baf2ad32ffc6a5ac0ede5c5efc475ec97d89e2883b1323de29408817ea80"
Nov 03 21:29:12.944306 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.944287    4993 scope.go:122] "RemoveContainer" containerID="51f6baf2ad32ffc6a5ac0ede5c5efc475ec97d89e2883b1323de29408817ea80"
Nov 03 21:29:12.944564 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:29:12.944545    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51f6baf2ad32ffc6a5ac0ede5c5efc475ec97d89e2883b1323de29408817ea80\": container with ID starting with 51f6baf2ad32ffc6a5ac0ede5c5efc475ec97d89e2883b1323de29408817ea80 not found: ID does not exist" containerID="51f6baf2ad32ffc6a5ac0ede5c5efc475ec97d89e2883b1323de29408817ea80"
Nov 03 21:29:12.944615 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.944577    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51f6baf2ad32ffc6a5ac0ede5c5efc475ec97d89e2883b1323de29408817ea80"} err="failed to get container status \"51f6baf2ad32ffc6a5ac0ede5c5efc475ec97d89e2883b1323de29408817ea80\": rpc error: code = NotFound desc = could not find container \"51f6baf2ad32ffc6a5ac0ede5c5efc475ec97d89e2883b1323de29408817ea80\": container with ID starting with 51f6baf2ad32ffc6a5ac0ede5c5efc475ec97d89e2883b1323de29408817ea80 not found: ID does not exist"
Nov 03 21:29:12.953840 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.953823    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-maas-api-tls\") pod \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") "
Nov 03 21:29:12.953909 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.953864    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-metrics-certs\") pod \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") "
Nov 03 21:29:12.953947 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.953924    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8551d21b-2481-4570-a9dd-57a3a01ba9e7-kube-api-access-gx9ls\" (UniqueName: \"kubernetes.io/projected/8551d21b-2481-4570-a9dd-57a3a01ba9e7-kube-api-access-gx9ls\") pod \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") "
Nov 03 21:29:12.954081 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.954061    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8551d21b-2481-4570-a9dd-57a3a01ba9e7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8551d21b-2481-4570-a9dd-57a3a01ba9e7-openshift-service-ca\") pod \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\" (UID: \"8551d21b-2481-4570-a9dd-57a3a01ba9e7\") "
Nov 03 21:29:12.954665 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.954520    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8551d21b-2481-4570-a9dd-57a3a01ba9e7-openshift-service-ca" pod "8551d21b-2481-4570-a9dd-57a3a01ba9e7" (UID: "8551d21b-2481-4570-a9dd-57a3a01ba9e7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:29:12.955995 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.955971    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-metrics-certs" pod "8551d21b-2481-4570-a9dd-57a3a01ba9e7" (UID: "8551d21b-2481-4570-a9dd-57a3a01ba9e7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:29:12.956167 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.956151    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8551d21b-2481-4570-a9dd-57a3a01ba9e7-kube-api-access-gx9ls" pod "8551d21b-2481-4570-a9dd-57a3a01ba9e7" (UID: "8551d21b-2481-4570-a9dd-57a3a01ba9e7"). InnerVolumeSpecName "kube-api-access-gx9ls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:29:12.956258 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:12.956238    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-maas-api-tls" pod "8551d21b-2481-4570-a9dd-57a3a01ba9e7" (UID: "8551d21b-2481-4570-a9dd-57a3a01ba9e7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:29:13.054999 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:13.054936    4993 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-metrics-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:13.054999 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:13.054961    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx9ls\" (UniqueName: \"kubernetes.io/projected/8551d21b-2481-4570-a9dd-57a3a01ba9e7-kube-api-access-gx9ls\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:13.054999 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:13.054970    4993 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8551d21b-2481-4570-a9dd-57a3a01ba9e7-openshift-service-ca\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:13.054999 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:13.054979    4993 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8551d21b-2481-4570-a9dd-57a3a01ba9e7-maas-api-tls\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:13.354713 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:13.354612    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"]
Nov 03 21:29:13.359684 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:13.359654    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-eb1249-6cf97d5765-tnvl2"]
Nov 03 21:29:13.576046 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:13.576014    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8551d21b-2481-4570-a9dd-57a3a01ba9e7" path="/var/lib/kubelet/pods/8551d21b-2481-4570-a9dd-57a3a01ba9e7/volumes"
Nov 03 21:29:13.716153 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:13.716113    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg" podUID="55e7133b-3b3d-4c59-a06e-d18883b6dca5" containerName="main" containerID="cri-o://e81cde2d6b05d7915bc0e113728cab7e5814805a98d7d5b7839fab1a216d3844" gracePeriod=45
Nov 03 21:29:13.942913 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:13.942880    4993 generic.go:356] "Generic (PLEG): container finished" podID="55e7133b-3b3d-4c59-a06e-d18883b6dca5" containerID="e81cde2d6b05d7915bc0e113728cab7e5814805a98d7d5b7839fab1a216d3844" exitCode=0
Nov 03 21:29:13.943409 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:13.942959    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg" event={"ID":"55e7133b-3b3d-4c59-a06e-d18883b6dca5","Type":"ContainerDied","Data":"e81cde2d6b05d7915bc0e113728cab7e5814805a98d7d5b7839fab1a216d3844"}
Nov 03 21:29:13.965671 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:13.965646    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:29:14.063555 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.063475    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-home\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-home\") pod \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") "
Nov 03 21:29:14.063741 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.063610    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-tmp-dir\") pod \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") "
Nov 03 21:29:14.063741 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.063685    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55e7133b-3b3d-4c59-a06e-d18883b6dca5-kube-api-access-w6wfg\" (UniqueName: \"kubernetes.io/projected/55e7133b-3b3d-4c59-a06e-d18883b6dca5-kube-api-access-w6wfg\") pod \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") "
Nov 03 21:29:14.063863 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.063752    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-model-cache\") pod \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") "
Nov 03 21:29:14.063863 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.063761    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-home" pod "55e7133b-3b3d-4c59-a06e-d18883b6dca5" (UID: "55e7133b-3b3d-4c59-a06e-d18883b6dca5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:29:14.063863 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.063798    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55e7133b-3b3d-4c59-a06e-d18883b6dca5-tls-certs\" (UniqueName: \"kubernetes.io/secret/55e7133b-3b3d-4c59-a06e-d18883b6dca5-tls-certs\") pod \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") "
Nov 03 21:29:14.063863 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.063827    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-dshm\") pod \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\" (UID: \"55e7133b-3b3d-4c59-a06e-d18883b6dca5\") "
Nov 03 21:29:14.063863 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.063848    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-tmp-dir" pod "55e7133b-3b3d-4c59-a06e-d18883b6dca5" (UID: "55e7133b-3b3d-4c59-a06e-d18883b6dca5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:29:14.064074 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.063952    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-model-cache" pod "55e7133b-3b3d-4c59-a06e-d18883b6dca5" (UID: "55e7133b-3b3d-4c59-a06e-d18883b6dca5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:29:14.064135 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.064116    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:14.064197 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.064138    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:14.064197 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.064147    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:14.066067 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.066046    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55e7133b-3b3d-4c59-a06e-d18883b6dca5-kube-api-access-w6wfg" pod "55e7133b-3b3d-4c59-a06e-d18883b6dca5" (UID: "55e7133b-3b3d-4c59-a06e-d18883b6dca5"). InnerVolumeSpecName "kube-api-access-w6wfg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:29:14.066156 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.066076    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55e7133b-3b3d-4c59-a06e-d18883b6dca5-tls-certs" pod "55e7133b-3b3d-4c59-a06e-d18883b6dca5" (UID: "55e7133b-3b3d-4c59-a06e-d18883b6dca5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:29:14.066156 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.066103    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-dshm" pod "55e7133b-3b3d-4c59-a06e-d18883b6dca5" (UID: "55e7133b-3b3d-4c59-a06e-d18883b6dca5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:29:14.164988 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.164949    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55e7133b-3b3d-4c59-a06e-d18883b6dca5-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:14.164988 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.164980    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55e7133b-3b3d-4c59-a06e-d18883b6dca5-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:14.164988 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.164992    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6wfg\" (UniqueName: \"kubernetes.io/projected/55e7133b-3b3d-4c59-a06e-d18883b6dca5-kube-api-access-w6wfg\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:14.949695 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.949660    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"
Nov 03 21:29:14.949695 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.949686    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg" event={"ID":"55e7133b-3b3d-4c59-a06e-d18883b6dca5","Type":"ContainerDied","Data":"a3c18c3b417160f806fbf4cd7cc5d9191d210538cd290b4857c9914c4c5ebb83"}
Nov 03 21:29:14.950176 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:14.949737    4993 scope.go:122] "RemoveContainer" containerID="e81cde2d6b05d7915bc0e113728cab7e5814805a98d7d5b7839fab1a216d3844"
Nov 03 21:29:15.111923 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:15.111893    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"]
Nov 03 21:29:15.161504 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:15.161479    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-eb1249/simulator-single-main-eb1249-kserve-76bdb7f649-zdvwg"]
Nov 03 21:29:15.575590 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:15.575557    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55e7133b-3b3d-4c59-a06e-d18883b6dca5" path="/var/lib/kubelet/pods/55e7133b-3b3d-4c59-a06e-d18883b6dca5/volumes"
Nov 03 21:29:20.863782 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:20.863748    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"]
Nov 03 21:29:20.864547 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:20.864515    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6" podUID="7450de41-5cbb-4aeb-8141-b71a01ca7189" containerName="istio-proxy" containerID="cri-o://59aa240593d783d294d1b2d905f95abda8e841ddef59aaa2eb39513564700c14" gracePeriod=30
Nov 03 21:29:26.001982 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.001946    4993 generic.go:356] "Generic (PLEG): container finished" podID="7450de41-5cbb-4aeb-8141-b71a01ca7189" containerID="59aa240593d783d294d1b2d905f95abda8e841ddef59aaa2eb39513564700c14" exitCode=0
Nov 03 21:29:26.002373 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.002024    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6" event={"ID":"7450de41-5cbb-4aeb-8141-b71a01ca7189","Type":"ContainerDied","Data":"59aa240593d783d294d1b2d905f95abda8e841ddef59aaa2eb39513564700c14"}
Nov 03 21:29:26.002373 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.002121    4993 scope.go:122] "RemoveContainer" containerID="c5dc9f89f6434c709fa075ab0f2609ce174bd3de034f7bdb86de0b25c32da5ae"
Nov 03 21:29:26.121352 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.121327    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:29:26.264654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.264610    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-token\" (UniqueName: \"kubernetes.io/projected/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-token\") pod \"7450de41-5cbb-4aeb-8141-b71a01ca7189\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") "
Nov 03 21:29:26.264813 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.264665    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-podinfo\") pod \"7450de41-5cbb-4aeb-8141-b71a01ca7189\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") "
Nov 03 21:29:26.264813 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.264709    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-credential-socket\") pod \"7450de41-5cbb-4aeb-8141-b71a01ca7189\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") "
Nov 03 21:29:26.264813 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.264749    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-envoy\") pod \"7450de41-5cbb-4aeb-8141-b71a01ca7189\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") "
Nov 03 21:29:26.264813 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.264781    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-workload-socket\") pod \"7450de41-5cbb-4aeb-8141-b71a01ca7189\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") "
Nov 03 21:29:26.264813 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.264806    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7450de41-5cbb-4aeb-8141-b71a01ca7189-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7450de41-5cbb-4aeb-8141-b71a01ca7189-istiod-ca-cert\") pod \"7450de41-5cbb-4aeb-8141-b71a01ca7189\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") "
Nov 03 21:29:26.265122 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.264841    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7450de41-5cbb-4aeb-8141-b71a01ca7189-kube-api-access-kqfl8\" (UniqueName: \"kubernetes.io/projected/7450de41-5cbb-4aeb-8141-b71a01ca7189-kube-api-access-kqfl8\") pod \"7450de41-5cbb-4aeb-8141-b71a01ca7189\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") "
Nov 03 21:29:26.265122 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.264957    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-workload-certs\") pod \"7450de41-5cbb-4aeb-8141-b71a01ca7189\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") "
Nov 03 21:29:26.265122 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.264988    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-data\") pod \"7450de41-5cbb-4aeb-8141-b71a01ca7189\" (UID: \"7450de41-5cbb-4aeb-8141-b71a01ca7189\") "
Nov 03 21:29:26.265122 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.265084    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-workload-socket" pod "7450de41-5cbb-4aeb-8141-b71a01ca7189" (UID: "7450de41-5cbb-4aeb-8141-b71a01ca7189"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:29:26.265332 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.265194    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-workload-certs" pod "7450de41-5cbb-4aeb-8141-b71a01ca7189" (UID: "7450de41-5cbb-4aeb-8141-b71a01ca7189"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:29:26.265332 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.265281    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-data" pod "7450de41-5cbb-4aeb-8141-b71a01ca7189" (UID: "7450de41-5cbb-4aeb-8141-b71a01ca7189"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:29:26.265332 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.265337    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:26.265667 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.265357    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:26.265667 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.265362    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-credential-socket" pod "7450de41-5cbb-4aeb-8141-b71a01ca7189" (UID: "7450de41-5cbb-4aeb-8141-b71a01ca7189"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:29:26.265785 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.265704    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7450de41-5cbb-4aeb-8141-b71a01ca7189-istiod-ca-cert" pod "7450de41-5cbb-4aeb-8141-b71a01ca7189" (UID: "7450de41-5cbb-4aeb-8141-b71a01ca7189"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:29:26.267048 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.267027    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-podinfo" pod "7450de41-5cbb-4aeb-8141-b71a01ca7189" (UID: "7450de41-5cbb-4aeb-8141-b71a01ca7189"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:29:26.267048 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.267031    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-token" pod "7450de41-5cbb-4aeb-8141-b71a01ca7189" (UID: "7450de41-5cbb-4aeb-8141-b71a01ca7189"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:29:26.267249 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.267229    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7450de41-5cbb-4aeb-8141-b71a01ca7189-kube-api-access-kqfl8" pod "7450de41-5cbb-4aeb-8141-b71a01ca7189" (UID: "7450de41-5cbb-4aeb-8141-b71a01ca7189"). InnerVolumeSpecName "kube-api-access-kqfl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:29:26.267527 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.267509    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-envoy" pod "7450de41-5cbb-4aeb-8141-b71a01ca7189" (UID: "7450de41-5cbb-4aeb-8141-b71a01ca7189"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:29:26.366362 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.366330    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:26.366362 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.366357    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:26.366362 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.366368    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:26.366556 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.366377    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:26.366556 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.366385    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7450de41-5cbb-4aeb-8141-b71a01ca7189-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:26.366556 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.366393    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqfl8\" (UniqueName: \"kubernetes.io/projected/7450de41-5cbb-4aeb-8141-b71a01ca7189-kube-api-access-kqfl8\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:26.366556 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:26.366401    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7450de41-5cbb-4aeb-8141-b71a01ca7189-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:29:27.008093 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:27.008053    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6" event={"ID":"7450de41-5cbb-4aeb-8141-b71a01ca7189","Type":"ContainerDied","Data":"e2634ce9adaab13b592613c6258903da19e72fcce5ab062cbeb1504bc6816bc2"}
Nov 03 21:29:27.008093 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:27.008086    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"
Nov 03 21:29:27.008533 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:27.008127    4993 scope.go:122] "RemoveContainer" containerID="59aa240593d783d294d1b2d905f95abda8e841ddef59aaa2eb39513564700c14"
Nov 03 21:29:27.221485 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:27.221452    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"]
Nov 03 21:29:27.271139 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:27.271080    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-single-main-eb1249-openshift-default-68655c749c99bs6"]
Nov 03 21:29:27.575069 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:27.574983    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7450de41-5cbb-4aeb-8141-b71a01ca7189" path="/var/lib/kubelet/pods/7450de41-5cbb-4aeb-8141-b71a01ca7189/volumes"
Nov 03 21:29:28.471324 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471292    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"]
Nov 03 21:29:28.471729 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471640    4993 cpu_manager.go:414] "containerMap: removing container" podUID="f2f7f5a4-95df-49d6-95aa-04062fbde687" containerName="payload-processing"
Nov 03 21:29:28.471729 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471652    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2f7f5a4-95df-49d6-95aa-04062fbde687" containerName="payload-processing"
Nov 03 21:29:28.471729 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471689    4993 cpu_manager.go:414] "containerMap: removing container" podUID="8551d21b-2481-4570-a9dd-57a3a01ba9e7" containerName="maas-api"
Nov 03 21:29:28.471729 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471694    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8551d21b-2481-4570-a9dd-57a3a01ba9e7" containerName="maas-api"
Nov 03 21:29:28.471729 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471706    4993 cpu_manager.go:414] "containerMap: removing container" podUID="7450de41-5cbb-4aeb-8141-b71a01ca7189" containerName="istio-proxy"
Nov 03 21:29:28.471729 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471711    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7450de41-5cbb-4aeb-8141-b71a01ca7189" containerName="istio-proxy"
Nov 03 21:29:28.471729 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471717    4993 cpu_manager.go:414] "containerMap: removing container" podUID="7450de41-5cbb-4aeb-8141-b71a01ca7189" containerName="istio-proxy"
Nov 03 21:29:28.471729 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471721    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7450de41-5cbb-4aeb-8141-b71a01ca7189" containerName="istio-proxy"
Nov 03 21:29:28.471729 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471729    4993 cpu_manager.go:414] "containerMap: removing container" podUID="55e7133b-3b3d-4c59-a06e-d18883b6dca5" containerName="main"
Nov 03 21:29:28.471729 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471735    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55e7133b-3b3d-4c59-a06e-d18883b6dca5" containerName="main"
Nov 03 21:29:28.472070 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471788    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="8551d21b-2481-4570-a9dd-57a3a01ba9e7" containerName="maas-api"
Nov 03 21:29:28.472070 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471797    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="7450de41-5cbb-4aeb-8141-b71a01ca7189" containerName="istio-proxy"
Nov 03 21:29:28.472070 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471804    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="7450de41-5cbb-4aeb-8141-b71a01ca7189" containerName="istio-proxy"
Nov 03 21:29:28.472070 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471810    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="55e7133b-3b3d-4c59-a06e-d18883b6dca5" containerName="main"
Nov 03 21:29:28.472070 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.471816    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2f7f5a4-95df-49d6-95aa-04062fbde687" containerName="payload-processing"
Nov 03 21:29:28.475939 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.475920    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.528149 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.528124    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-8eba094a-kserve-self-signed-certs\""
Nov 03 21:29:28.577739 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.577711    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Nov 03 21:29:28.577739 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.577711    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Nov 03 21:29:28.578968 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.578948    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"]
Nov 03 21:29:28.585058 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.585030    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-model-cache\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.585149 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.585069    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-dshm\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.585185 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.585152    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx9ms\" (UniqueName: \"kubernetes.io/projected/800d7c88-22f9-43cd-90c6-6aefdeeaa752-kube-api-access-fx9ms\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.585238 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.585224    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-tmp-dir\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.585276 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.585264    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/800d7c88-22f9-43cd-90c6-6aefdeeaa752-tls-certs\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.585313 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.585296    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-home\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.628396 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.628367    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wnljc\""
Nov 03 21:29:28.686709 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.686677    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx9ms\" (UniqueName: \"kubernetes.io/projected/800d7c88-22f9-43cd-90c6-6aefdeeaa752-kube-api-access-fx9ms\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.686882 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.686718    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-tmp-dir\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.686882 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.686777    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/800d7c88-22f9-43cd-90c6-6aefdeeaa752-tls-certs\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.686882 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.686820    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-home\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.686882 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.686869    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-model-cache\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.687164 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.686898    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-dshm\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.687164 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.687132    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-tmp-dir\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.687272 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.687200    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-home\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.687272 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.687240    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-model-cache\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.689458 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.689431    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-dshm\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.689982 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.689961    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/800d7c88-22f9-43cd-90c6-6aefdeeaa752-tls-certs\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.747907 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.747839    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx9ms\" (UniqueName: \"kubernetes.io/projected/800d7c88-22f9-43cd-90c6-6aefdeeaa752-kube-api-access-fx9ms\") pod \"e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") " pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:28.801697 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:28.801661    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:29.181661 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:29.181618    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"]
Nov 03 21:29:29.184216 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:29:29.184191    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod800d7c88_22f9_43cd_90c6_6aefdeeaa752.slice/crio-7cc5628a71dab556315b149ae8da03b0c2366cdfba323df02a29aecb98dd0216 WatchSource:0}: Error finding container 7cc5628a71dab556315b149ae8da03b0c2366cdfba323df02a29aecb98dd0216: Status 404 returned error can't find the container with id 7cc5628a71dab556315b149ae8da03b0c2366cdfba323df02a29aecb98dd0216
Nov 03 21:29:30.023900 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:30.023862    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l" event={"ID":"800d7c88-22f9-43cd-90c6-6aefdeeaa752","Type":"ContainerStarted","Data":"04df7c47658f610f74381618d965d62fa83333511adb488b4a86ca1f909173e2"}
Nov 03 21:29:30.024260 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:30.023908    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l" event={"ID":"800d7c88-22f9-43cd-90c6-6aefdeeaa752","Type":"ContainerStarted","Data":"7cc5628a71dab556315b149ae8da03b0c2366cdfba323df02a29aecb98dd0216"}
Nov 03 21:29:30.144344 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:30.141904    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l" podStartSLOduration=2.141885923 podStartE2EDuration="2.141885923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:29:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:29:30.139520492 +0000 UTC m=+2972.684226862" watchObservedRunningTime="2026-11-03 21:29:30.141885923 +0000 UTC m=+2972.686592293"
Nov 03 21:29:30.801952 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:30.801920    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:31.668682 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:29:31.668618    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:29:38.802479 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:38.802397    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:38.814948 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:38.814925    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:29:39.079738 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:29:39.079669    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:30:01.669133 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:30:01.669096    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:30:13.297675 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.297641    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"]
Nov 03 21:30:13.302498 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.302476    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.376246 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.376206    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-home\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.376441 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.376291    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aebcaffc-db3b-4b38-a84a-a60d7af210c4-tls-certs\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.376441 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.376323    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-dshm\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.376441 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.376396    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-model-cache\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.376441 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.376437    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-tmp-dir\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.376591 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.376529    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lljn2\" (UniqueName: \"kubernetes.io/projected/aebcaffc-db3b-4b38-a84a-a60d7af210c4-kube-api-access-lljn2\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.404136 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.404097    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-8eba094a-kserve-self-signed-certs\""
Nov 03 21:30:13.405520 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.405500    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"]
Nov 03 21:30:13.477299 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.477264    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aebcaffc-db3b-4b38-a84a-a60d7af210c4-tls-certs\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.477477 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.477307    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-dshm\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.477477 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.477342    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-model-cache\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.477477 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.477361    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-tmp-dir\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.477477 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.477381    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lljn2\" (UniqueName: \"kubernetes.io/projected/aebcaffc-db3b-4b38-a84a-a60d7af210c4-kube-api-access-lljn2\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.477477 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.477413    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-home\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.477811 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.477789    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-tmp-dir\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.477887 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.477826    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-home\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.477887 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.477874    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-model-cache\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.479822 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.479799    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-dshm\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.479938 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.479923    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aebcaffc-db3b-4b38-a84a-a60d7af210c4-tls-certs\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.537842 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.537812    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lljn2\" (UniqueName: \"kubernetes.io/projected/aebcaffc-db3b-4b38-a84a-a60d7af210c4-kube-api-access-lljn2\") pod \"e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") " pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.614018 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.613947    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:13.987305 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:13.987273    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"]
Nov 03 21:30:13.989922 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:30:13.989895    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaebcaffc_db3b_4b38_a84a_a60d7af210c4.slice/crio-97c103a95c4b28ef2ae3f6c0cdab9db3ec2f9a6cf6401ffe007090496f9e8eb2 WatchSource:0}: Error finding container 97c103a95c4b28ef2ae3f6c0cdab9db3ec2f9a6cf6401ffe007090496f9e8eb2: Status 404 returned error can't find the container with id 97c103a95c4b28ef2ae3f6c0cdab9db3ec2f9a6cf6401ffe007090496f9e8eb2
Nov 03 21:30:14.224316 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:14.224279    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w" event={"ID":"aebcaffc-db3b-4b38-a84a-a60d7af210c4","Type":"ContainerStarted","Data":"053b66f5a8e555621869e357ef10cefe8fed674211546fdc7d7e0b6410e52d0a"}
Nov 03 21:30:14.224501 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:14.224327    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w" event={"ID":"aebcaffc-db3b-4b38-a84a-a60d7af210c4","Type":"ContainerStarted","Data":"97c103a95c4b28ef2ae3f6c0cdab9db3ec2f9a6cf6401ffe007090496f9e8eb2"}
Nov 03 21:30:14.340796 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:14.340700    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w" podStartSLOduration=1.340686263 podStartE2EDuration="1.340686263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:30:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:30:14.339139314 +0000 UTC m=+3016.883845684" watchObservedRunningTime="2026-11-03 21:30:14.340686263 +0000 UTC m=+3016.885392627"
Nov 03 21:30:14.614604 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:14.614526    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:23.615083 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:23.615049    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:23.627220 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:23.627190    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:24.280568 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:24.280531    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:30:31.670659 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:30:31.670608    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:30:48.977867 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:48.977775    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:30:48.978598 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:48.978556    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-g24jg" podUID="ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:30:49.129933 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:49.129894    4993 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 03 21:30:49.130268 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:49.130238    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7" podUID="b056b057-0700-4450-b07d-f19b53d5584a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 03 21:30:56.727744 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:30:56.727711    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"]
Nov 03 21:31:01.736928 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:31:01.736881    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:31:11.747736 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:11.747617    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l" podUID="800d7c88-22f9-43cd-90c6-6aefdeeaa752" containerName="main" containerID="cri-o://04df7c47658f610f74381618d965d62fa83333511adb488b4a86ca1f909173e2" gracePeriod=45
Nov 03 21:31:11.994726 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:11.994701    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:31:12.034259 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.034184    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-tmp-dir\") pod \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") "
Nov 03 21:31:12.034259 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.034241    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-dshm\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-dshm\") pod \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") "
Nov 03 21:31:12.034473 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.034288    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-model-cache\") pod \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") "
Nov 03 21:31:12.034473 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.034328    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/800d7c88-22f9-43cd-90c6-6aefdeeaa752-tls-certs\" (UniqueName: \"kubernetes.io/secret/800d7c88-22f9-43cd-90c6-6aefdeeaa752-tls-certs\") pod \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") "
Nov 03 21:31:12.034473 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.034384    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/800d7c88-22f9-43cd-90c6-6aefdeeaa752-kube-api-access-fx9ms\" (UniqueName: \"kubernetes.io/projected/800d7c88-22f9-43cd-90c6-6aefdeeaa752-kube-api-access-fx9ms\") pod \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") "
Nov 03 21:31:12.034473 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.034416    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-home\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-home\") pod \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\" (UID: \"800d7c88-22f9-43cd-90c6-6aefdeeaa752\") "
Nov 03 21:31:12.034473 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.034448    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-tmp-dir" pod "800d7c88-22f9-43cd-90c6-6aefdeeaa752" (UID: "800d7c88-22f9-43cd-90c6-6aefdeeaa752"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:31:12.034747 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.034547    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-model-cache" pod "800d7c88-22f9-43cd-90c6-6aefdeeaa752" (UID: "800d7c88-22f9-43cd-90c6-6aefdeeaa752"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:31:12.034747 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.034703    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-home" pod "800d7c88-22f9-43cd-90c6-6aefdeeaa752" (UID: "800d7c88-22f9-43cd-90c6-6aefdeeaa752"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:31:12.034850 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.034771    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:12.034850 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.034783    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:12.034850 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.034792    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:12.036745 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.036717    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/800d7c88-22f9-43cd-90c6-6aefdeeaa752-kube-api-access-fx9ms" pod "800d7c88-22f9-43cd-90c6-6aefdeeaa752" (UID: "800d7c88-22f9-43cd-90c6-6aefdeeaa752"). InnerVolumeSpecName "kube-api-access-fx9ms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:31:12.036745 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.036725    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/800d7c88-22f9-43cd-90c6-6aefdeeaa752-tls-certs" pod "800d7c88-22f9-43cd-90c6-6aefdeeaa752" (UID: "800d7c88-22f9-43cd-90c6-6aefdeeaa752"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:31:12.036926 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.036721    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-dshm" pod "800d7c88-22f9-43cd-90c6-6aefdeeaa752" (UID: "800d7c88-22f9-43cd-90c6-6aefdeeaa752"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:31:12.135837 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.135806    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/800d7c88-22f9-43cd-90c6-6aefdeeaa752-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:12.135837 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.135833    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/800d7c88-22f9-43cd-90c6-6aefdeeaa752-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:12.135837 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.135844    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fx9ms\" (UniqueName: \"kubernetes.io/projected/800d7c88-22f9-43cd-90c6-6aefdeeaa752-kube-api-access-fx9ms\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:12.503595 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.503562    4993 generic.go:356] "Generic (PLEG): container finished" podID="800d7c88-22f9-43cd-90c6-6aefdeeaa752" containerID="04df7c47658f610f74381618d965d62fa83333511adb488b4a86ca1f909173e2" exitCode=0
Nov 03 21:31:12.503778 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.503608    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"
Nov 03 21:31:12.503778 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.503655    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l" event={"ID":"800d7c88-22f9-43cd-90c6-6aefdeeaa752","Type":"ContainerDied","Data":"04df7c47658f610f74381618d965d62fa83333511adb488b4a86ca1f909173e2"}
Nov 03 21:31:12.503778 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.503702    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l" event={"ID":"800d7c88-22f9-43cd-90c6-6aefdeeaa752","Type":"ContainerDied","Data":"7cc5628a71dab556315b149ae8da03b0c2366cdfba323df02a29aecb98dd0216"}
Nov 03 21:31:12.503778 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.503724    4993 scope.go:122] "RemoveContainer" containerID="04df7c47658f610f74381618d965d62fa83333511adb488b4a86ca1f909173e2"
Nov 03 21:31:12.513402 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.513385    4993 scope.go:122] "RemoveContainer" containerID="04df7c47658f610f74381618d965d62fa83333511adb488b4a86ca1f909173e2"
Nov 03 21:31:12.513778 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:31:12.513760    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04df7c47658f610f74381618d965d62fa83333511adb488b4a86ca1f909173e2\": container with ID starting with 04df7c47658f610f74381618d965d62fa83333511adb488b4a86ca1f909173e2 not found: ID does not exist" containerID="04df7c47658f610f74381618d965d62fa83333511adb488b4a86ca1f909173e2"
Nov 03 21:31:12.513833 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.513788    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04df7c47658f610f74381618d965d62fa83333511adb488b4a86ca1f909173e2"} err="failed to get container status \"04df7c47658f610f74381618d965d62fa83333511adb488b4a86ca1f909173e2\": rpc error: code = NotFound desc = could not find container \"04df7c47658f610f74381618d965d62fa83333511adb488b4a86ca1f909173e2\": container with ID starting with 04df7c47658f610f74381618d965d62fa83333511adb488b4a86ca1f909173e2 not found: ID does not exist"
Nov 03 21:31:12.687360 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.687324    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"]
Nov 03 21:31:12.694608 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:12.694580    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-8eba094a-kserve-78f8d9d6b5-f5h9l"]
Nov 03 21:31:13.000259 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:13.000221    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"]
Nov 03 21:31:13.575740 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:13.575708    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="800d7c88-22f9-43cd-90c6-6aefdeeaa752" path="/var/lib/kubelet/pods/800d7c88-22f9-43cd-90c6-6aefdeeaa752/volumes"
Nov 03 21:31:28.020887 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.020821    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w" podUID="aebcaffc-db3b-4b38-a84a-a60d7af210c4" containerName="main" containerID="cri-o://053b66f5a8e555621869e357ef10cefe8fed674211546fdc7d7e0b6410e52d0a" gracePeriod=45
Nov 03 21:31:28.265426 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.265402    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:31:28.307911 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.307853    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-tmp-dir\") pod \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") "
Nov 03 21:31:28.307911 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.307888    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-model-cache\") pod \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") "
Nov 03 21:31:28.308062 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.307916    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aebcaffc-db3b-4b38-a84a-a60d7af210c4-tls-certs\" (UniqueName: \"kubernetes.io/secret/aebcaffc-db3b-4b38-a84a-a60d7af210c4-tls-certs\") pod \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") "
Nov 03 21:31:28.308062 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.307964    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-dshm\") pod \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") "
Nov 03 21:31:28.308062 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.308033    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aebcaffc-db3b-4b38-a84a-a60d7af210c4-kube-api-access-lljn2\" (UniqueName: \"kubernetes.io/projected/aebcaffc-db3b-4b38-a84a-a60d7af210c4-kube-api-access-lljn2\") pod \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") "
Nov 03 21:31:28.308224 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.308090    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-home\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-home\") pod \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\" (UID: \"aebcaffc-db3b-4b38-a84a-a60d7af210c4\") "
Nov 03 21:31:28.308224 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.308129    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-model-cache" pod "aebcaffc-db3b-4b38-a84a-a60d7af210c4" (UID: "aebcaffc-db3b-4b38-a84a-a60d7af210c4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:31:28.308224 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.308144    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-tmp-dir" pod "aebcaffc-db3b-4b38-a84a-a60d7af210c4" (UID: "aebcaffc-db3b-4b38-a84a-a60d7af210c4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:31:28.308388 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.308337    4993 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-tmp-dir\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:28.308388 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.308360    4993 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-model-cache\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:28.308486 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.308392    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-home" pod "aebcaffc-db3b-4b38-a84a-a60d7af210c4" (UID: "aebcaffc-db3b-4b38-a84a-a60d7af210c4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:31:28.310172 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.310142    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aebcaffc-db3b-4b38-a84a-a60d7af210c4-tls-certs" pod "aebcaffc-db3b-4b38-a84a-a60d7af210c4" (UID: "aebcaffc-db3b-4b38-a84a-a60d7af210c4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:31:28.310268 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.310169    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-dshm" pod "aebcaffc-db3b-4b38-a84a-a60d7af210c4" (UID: "aebcaffc-db3b-4b38-a84a-a60d7af210c4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:31:28.310268 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.310190    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aebcaffc-db3b-4b38-a84a-a60d7af210c4-kube-api-access-lljn2" pod "aebcaffc-db3b-4b38-a84a-a60d7af210c4" (UID: "aebcaffc-db3b-4b38-a84a-a60d7af210c4"). InnerVolumeSpecName "kube-api-access-lljn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:31:28.409406 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.409381    4993 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-dshm\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:28.409406 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.409407    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lljn2\" (UniqueName: \"kubernetes.io/projected/aebcaffc-db3b-4b38-a84a-a60d7af210c4-kube-api-access-lljn2\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:28.409587 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.409417    4993 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aebcaffc-db3b-4b38-a84a-a60d7af210c4-home\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:28.409587 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.409426    4993 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aebcaffc-db3b-4b38-a84a-a60d7af210c4-tls-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:31:28.582069 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.581995    4993 generic.go:356] "Generic (PLEG): container finished" podID="aebcaffc-db3b-4b38-a84a-a60d7af210c4" containerID="053b66f5a8e555621869e357ef10cefe8fed674211546fdc7d7e0b6410e52d0a" exitCode=0
Nov 03 21:31:28.582069 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.582039    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"
Nov 03 21:31:28.582226 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.582079    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w" event={"ID":"aebcaffc-db3b-4b38-a84a-a60d7af210c4","Type":"ContainerDied","Data":"053b66f5a8e555621869e357ef10cefe8fed674211546fdc7d7e0b6410e52d0a"}
Nov 03 21:31:28.582226 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.582124    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w" event={"ID":"aebcaffc-db3b-4b38-a84a-a60d7af210c4","Type":"ContainerDied","Data":"97c103a95c4b28ef2ae3f6c0cdab9db3ec2f9a6cf6401ffe007090496f9e8eb2"}
Nov 03 21:31:28.582226 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.582142    4993 scope.go:122] "RemoveContainer" containerID="053b66f5a8e555621869e357ef10cefe8fed674211546fdc7d7e0b6410e52d0a"
Nov 03 21:31:28.591852 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.591834    4993 scope.go:122] "RemoveContainer" containerID="053b66f5a8e555621869e357ef10cefe8fed674211546fdc7d7e0b6410e52d0a"
Nov 03 21:31:28.592100 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:31:28.592081    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"053b66f5a8e555621869e357ef10cefe8fed674211546fdc7d7e0b6410e52d0a\": container with ID starting with 053b66f5a8e555621869e357ef10cefe8fed674211546fdc7d7e0b6410e52d0a not found: ID does not exist" containerID="053b66f5a8e555621869e357ef10cefe8fed674211546fdc7d7e0b6410e52d0a"
Nov 03 21:31:28.592156 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.592112    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"053b66f5a8e555621869e357ef10cefe8fed674211546fdc7d7e0b6410e52d0a"} err="failed to get container status \"053b66f5a8e555621869e357ef10cefe8fed674211546fdc7d7e0b6410e52d0a\": rpc error: code = NotFound desc = could not find container \"053b66f5a8e555621869e357ef10cefe8fed674211546fdc7d7e0b6410e52d0a\": container with ID starting with 053b66f5a8e555621869e357ef10cefe8fed674211546fdc7d7e0b6410e52d0a not found: ID does not exist"
Nov 03 21:31:28.744452 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.744422    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"]
Nov 03 21:31:28.748226 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:28.748203    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-8eba094a-kserve-59b466f54f-gwc5w"]
Nov 03 21:31:29.581539 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:31:29.581498    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aebcaffc-db3b-4b38-a84a-a60d7af210c4" path="/var/lib/kubelet/pods/aebcaffc-db3b-4b38-a84a-a60d7af210c4/volumes"
Nov 03 21:31:31.742352 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:31:31.742309    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:32:01.669065 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:32:01.669029    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:32:07.784310 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:07.784271    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-76d76db597-cthjw"]
Nov 03 21:32:07.784870 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:07.784846    4993 cpu_manager.go:414] "containerMap: removing container" podUID="aebcaffc-db3b-4b38-a84a-a60d7af210c4" containerName="main"
Nov 03 21:32:07.784927 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:07.784873    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aebcaffc-db3b-4b38-a84a-a60d7af210c4" containerName="main"
Nov 03 21:32:07.784927 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:07.784900    4993 cpu_manager.go:414] "containerMap: removing container" podUID="800d7c88-22f9-43cd-90c6-6aefdeeaa752" containerName="main"
Nov 03 21:32:07.784927 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:07.784908    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="800d7c88-22f9-43cd-90c6-6aefdeeaa752" containerName="main"
Nov 03 21:32:07.785023 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:07.784999    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="aebcaffc-db3b-4b38-a84a-a60d7af210c4" containerName="main"
Nov 03 21:32:07.785023 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:07.785013    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="800d7c88-22f9-43cd-90c6-6aefdeeaa752" containerName="main"
Nov 03 21:32:07.789607 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:07.789586    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:07.893860 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:07.893819    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76d76db597-cthjw"]
Nov 03 21:32:07.940060 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:07.940025    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97a2836b-4d21-42ab-b233-fba1a3cc887c-metrics-certs\") pod \"maas-api-76d76db597-cthjw\" (UID: \"97a2836b-4d21-42ab-b233-fba1a3cc887c\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:07.940267 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:07.940067    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97a2836b-4d21-42ab-b233-fba1a3cc887c-openshift-service-ca\") pod \"maas-api-76d76db597-cthjw\" (UID: \"97a2836b-4d21-42ab-b233-fba1a3cc887c\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:07.940267 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:07.940198    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97a2836b-4d21-42ab-b233-fba1a3cc887c-maas-api-tls\") pod \"maas-api-76d76db597-cthjw\" (UID: \"97a2836b-4d21-42ab-b233-fba1a3cc887c\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:07.940390 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:07.940265    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbnzq\" (UniqueName: \"kubernetes.io/projected/97a2836b-4d21-42ab-b233-fba1a3cc887c-kube-api-access-hbnzq\") pod \"maas-api-76d76db597-cthjw\" (UID: \"97a2836b-4d21-42ab-b233-fba1a3cc887c\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:08.041464 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:08.041382    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97a2836b-4d21-42ab-b233-fba1a3cc887c-metrics-certs\") pod \"maas-api-76d76db597-cthjw\" (UID: \"97a2836b-4d21-42ab-b233-fba1a3cc887c\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:08.041464 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:08.041434    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97a2836b-4d21-42ab-b233-fba1a3cc887c-openshift-service-ca\") pod \"maas-api-76d76db597-cthjw\" (UID: \"97a2836b-4d21-42ab-b233-fba1a3cc887c\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:08.041737 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:08.041504    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97a2836b-4d21-42ab-b233-fba1a3cc887c-maas-api-tls\") pod \"maas-api-76d76db597-cthjw\" (UID: \"97a2836b-4d21-42ab-b233-fba1a3cc887c\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:08.041737 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:08.041547    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbnzq\" (UniqueName: \"kubernetes.io/projected/97a2836b-4d21-42ab-b233-fba1a3cc887c-kube-api-access-hbnzq\") pod \"maas-api-76d76db597-cthjw\" (UID: \"97a2836b-4d21-42ab-b233-fba1a3cc887c\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:08.042183 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:08.042160    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97a2836b-4d21-42ab-b233-fba1a3cc887c-openshift-service-ca\") pod \"maas-api-76d76db597-cthjw\" (UID: \"97a2836b-4d21-42ab-b233-fba1a3cc887c\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:08.044076 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:08.044047    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97a2836b-4d21-42ab-b233-fba1a3cc887c-metrics-certs\") pod \"maas-api-76d76db597-cthjw\" (UID: \"97a2836b-4d21-42ab-b233-fba1a3cc887c\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:08.044268 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:08.044247    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97a2836b-4d21-42ab-b233-fba1a3cc887c-maas-api-tls\") pod \"maas-api-76d76db597-cthjw\" (UID: \"97a2836b-4d21-42ab-b233-fba1a3cc887c\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:08.102636 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:08.102602    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbnzq\" (UniqueName: \"kubernetes.io/projected/97a2836b-4d21-42ab-b233-fba1a3cc887c-kube-api-access-hbnzq\") pod \"maas-api-76d76db597-cthjw\" (UID: \"97a2836b-4d21-42ab-b233-fba1a3cc887c\") " pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:08.103219 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:08.103203    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:08.473863 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:08.473838    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-76d76db597-cthjw"]
Nov 03 21:32:08.476414 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:32:08.476390    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97a2836b_4d21_42ab_b233_fba1a3cc887c.slice/crio-c3d57d14dcf34c7b1ea65f3146f9f7255bead5c549f52482f91e291b7d4e9e4f WatchSource:0}: Error finding container c3d57d14dcf34c7b1ea65f3146f9f7255bead5c549f52482f91e291b7d4e9e4f: Status 404 returned error can't find the container with id c3d57d14dcf34c7b1ea65f3146f9f7255bead5c549f52482f91e291b7d4e9e4f
Nov 03 21:32:08.477716 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:08.477694    4993 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 21:32:08.808214 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:08.808119    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw" event={"ID":"97a2836b-4d21-42ab-b233-fba1a3cc887c","Type":"ContainerStarted","Data":"c3d57d14dcf34c7b1ea65f3146f9f7255bead5c549f52482f91e291b7d4e9e4f"}
Nov 03 21:32:09.813725 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:09.813687    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw" event={"ID":"97a2836b-4d21-42ab-b233-fba1a3cc887c","Type":"ContainerStarted","Data":"d84c4436547ed01ae7900c5885191781a794e19b5a4b6dc6ce28d21e88d72ab9"}
Nov 03 21:32:09.814155 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:09.813815    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:09.931437 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:09.931385    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw" podStartSLOduration=2.08628372 podStartE2EDuration="2.931369932s" totalImagesPullingTime="845.086212ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:32:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:32:09.930096971 +0000 UTC m=+3132.474803344" watchObservedRunningTime="2026-11-03 21:32:09.931369932 +0000 UTC m=+3132.476076303"
Nov 03 21:32:15.824335 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:15.824307    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-76d76db597-cthjw"
Nov 03 21:32:24.177825 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.177791    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"]
Nov 03 21:32:24.178282 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.178028    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m" podUID="8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb" containerName="maas-api" containerID="cri-o://4c4e1ff46bad4f22cccefae49dc83d9ccc7234685d653c51719c68de3756e362" gracePeriod=30
Nov 03 21:32:24.703120 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.703089    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"]
Nov 03 21:32:24.706587 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.706569    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.759110 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.759075    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-bb2b291b-openshift-default-dockercfg-djnqb\""
Nov 03 21:32:24.781720 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.781686    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deb43ff0-9d5c-4117-9d0b-324769219775-istio-token\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.782099 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.781728    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-workload-certs\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.782099 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.781832    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-istio-envoy\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.782099 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.781879    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-credential-socket\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.782099 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.781910    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-istio-data\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.782099 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.781983    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/deb43ff0-9d5c-4117-9d0b-324769219775-istiod-ca-cert\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.782099 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.782011    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-workload-socket\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.782446 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.782112    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/deb43ff0-9d5c-4117-9d0b-324769219775-istio-podinfo\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.782446 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.782187    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbfbb\" (UniqueName: \"kubernetes.io/projected/deb43ff0-9d5c-4117-9d0b-324769219775-kube-api-access-jbfbb\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.810198 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.810165    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"]
Nov 03 21:32:24.883425 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.883393    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/deb43ff0-9d5c-4117-9d0b-324769219775-istio-podinfo\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.883575 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.883448    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbfbb\" (UniqueName: \"kubernetes.io/projected/deb43ff0-9d5c-4117-9d0b-324769219775-kube-api-access-jbfbb\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.883575 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.883519    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deb43ff0-9d5c-4117-9d0b-324769219775-istio-token\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.883681 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.883660    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-workload-certs\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.883713 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.883690    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-istio-envoy\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.883751 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.883718    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-credential-socket\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.883790 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.883745    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-istio-data\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.883790 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.883785    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/deb43ff0-9d5c-4117-9d0b-324769219775-istiod-ca-cert\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.883904 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.883827    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-workload-socket\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.884167 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.884138    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-workload-certs\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.884323 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.884191    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-istio-data\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.884323 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.884246    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-credential-socket\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.884323 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.884285    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-workload-socket\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.884843 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.884818    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/deb43ff0-9d5c-4117-9d0b-324769219775-istiod-ca-cert\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.886063 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.886034    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-istio-envoy\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.886239 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.886223    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/deb43ff0-9d5c-4117-9d0b-324769219775-istio-podinfo\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.944409 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.944386    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deb43ff0-9d5c-4117-9d0b-324769219775-istio-token\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:24.944575 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:24.944554    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbfbb\" (UniqueName: \"kubernetes.io/projected/deb43ff0-9d5c-4117-9d0b-324769219775-kube-api-access-jbfbb\") pod \"e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") " pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:25.018783 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.018755    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:25.232206 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.232176    4993 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m" podUID="8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.27:8443/health\": dial tcp 10.133.0.27:8443: connect: connection refused"
Nov 03 21:32:25.416015 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.415990    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 21:32:25.445742 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.445719    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"]
Nov 03 21:32:25.447567 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:32:25.447533    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeb43ff0_9d5c_4117_9d0b_324769219775.slice/crio-53dc90fbe3bb935fc09349f2bd1b561598a330b0f83f74321b4618726e4d0d3a WatchSource:0}: Error finding container 53dc90fbe3bb935fc09349f2bd1b561598a330b0f83f74321b4618726e4d0d3a: Status 404 returned error can't find the container with id 53dc90fbe3bb935fc09349f2bd1b561598a330b0f83f74321b4618726e4d0d3a
Nov 03 21:32:25.449655 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.449611    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:32:25.449723 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.449692    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:32:25.449761 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.449718    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:32:25.490041 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.490013    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-kube-api-access-8xvdt\" (UniqueName: \"kubernetes.io/projected/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-kube-api-access-8xvdt\") pod \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") "
Nov 03 21:32:25.490162 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.490108    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-metrics-certs\") pod \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") "
Nov 03 21:32:25.490162 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.490136    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-maas-api-tls\") pod \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") "
Nov 03 21:32:25.490162 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.490155    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-openshift-service-ca\") pod \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\" (UID: \"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb\") "
Nov 03 21:32:25.491352 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.490668    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-openshift-service-ca" pod "8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb" (UID: "8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:32:25.492469 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.492442    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-kube-api-access-8xvdt" pod "8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb" (UID: "8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb"). InnerVolumeSpecName "kube-api-access-8xvdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:32:25.492603 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.492554    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-maas-api-tls" pod "8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb" (UID: "8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:32:25.492677 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.492594    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-metrics-certs" pod "8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb" (UID: "8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:32:25.590740 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.590714    4993 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-metrics-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:32:25.590740 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.590741    4993 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-maas-api-tls\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:32:25.590900 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.590756    4993 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-openshift-service-ca\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:32:25.590900 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.590769    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xvdt\" (UniqueName: \"kubernetes.io/projected/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb-kube-api-access-8xvdt\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:32:25.598094 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:32:25.598067    4993 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd5f959_4f8f_4f7b_a394_be2ad7d1eecb.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fd5f959_4f8f_4f7b_a394_be2ad7d1eecb.slice/crio-fc0a5dfa34f7307d2a0c8610f7dc5fc6d55245e297d0cd5fb898826b90da2e45\": RecentStats: unable to find data in memory cache]"
Nov 03 21:32:25.888405 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.888329    4993 generic.go:356] "Generic (PLEG): container finished" podID="8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb" containerID="4c4e1ff46bad4f22cccefae49dc83d9ccc7234685d653c51719c68de3756e362" exitCode=0
Nov 03 21:32:25.888574 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.888405    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"
Nov 03 21:32:25.888574 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.888410    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m" event={"ID":"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb","Type":"ContainerDied","Data":"4c4e1ff46bad4f22cccefae49dc83d9ccc7234685d653c51719c68de3756e362"}
Nov 03 21:32:25.888574 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.888488    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8669f899bf-59f8m" event={"ID":"8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb","Type":"ContainerDied","Data":"fc0a5dfa34f7307d2a0c8610f7dc5fc6d55245e297d0cd5fb898826b90da2e45"}
Nov 03 21:32:25.888574 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.888519    4993 scope.go:122] "RemoveContainer" containerID="4c4e1ff46bad4f22cccefae49dc83d9ccc7234685d653c51719c68de3756e362"
Nov 03 21:32:25.890365 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.890334    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t" event={"ID":"deb43ff0-9d5c-4117-9d0b-324769219775","Type":"ContainerStarted","Data":"52e4608153d4cb3133dbd043b420be35899956ce0b343f2cd7e0d8a7b1c745c9"}
Nov 03 21:32:25.890469 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.890388    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t" event={"ID":"deb43ff0-9d5c-4117-9d0b-324769219775","Type":"ContainerStarted","Data":"53dc90fbe3bb935fc09349f2bd1b561598a330b0f83f74321b4618726e4d0d3a"}
Nov 03 21:32:25.906021 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.906005    4993 scope.go:122] "RemoveContainer" containerID="4c4e1ff46bad4f22cccefae49dc83d9ccc7234685d653c51719c68de3756e362"
Nov 03 21:32:25.906236 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:32:25.906217    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c4e1ff46bad4f22cccefae49dc83d9ccc7234685d653c51719c68de3756e362\": container with ID starting with 4c4e1ff46bad4f22cccefae49dc83d9ccc7234685d653c51719c68de3756e362 not found: ID does not exist" containerID="4c4e1ff46bad4f22cccefae49dc83d9ccc7234685d653c51719c68de3756e362"
Nov 03 21:32:25.906289 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:25.906245    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c4e1ff46bad4f22cccefae49dc83d9ccc7234685d653c51719c68de3756e362"} err="failed to get container status \"4c4e1ff46bad4f22cccefae49dc83d9ccc7234685d653c51719c68de3756e362\": rpc error: code = NotFound desc = could not find container \"4c4e1ff46bad4f22cccefae49dc83d9ccc7234685d653c51719c68de3756e362\": container with ID starting with 4c4e1ff46bad4f22cccefae49dc83d9ccc7234685d653c51719c68de3756e362 not found: ID does not exist"
Nov 03 21:32:26.019868 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:26.019830    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:26.024647 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:26.024610    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:26.059946 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:26.059900    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t" podStartSLOduration=2.059889465 podStartE2EDuration="2.059889465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:32:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:32:26.059155048 +0000 UTC m=+3148.603861419" watchObservedRunningTime="2026-11-03 21:32:26.059889465 +0000 UTC m=+3148.604595915"
Nov 03 21:32:26.169704 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:26.169620    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"]
Nov 03 21:32:26.173338 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:26.173316    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-8669f899bf-59f8m"]
Nov 03 21:32:26.896070 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:26.896034    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:26.896926 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:26.896908    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:27.576857 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:27.576825    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb" path="/var/lib/kubelet/pods/8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb/volumes"
Nov 03 21:32:31.668709 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:32:31.668673    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:32:32.038945 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.038909    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"]
Nov 03 21:32:32.039352 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.039322    4993 cpu_manager.go:414] "containerMap: removing container" podUID="8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb" containerName="maas-api"
Nov 03 21:32:32.039483 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.039356    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb" containerName="maas-api"
Nov 03 21:32:32.039569 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.039508    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="8fd5f959-4f8f-4f7b-a394-be2ad7d1eecb" containerName="maas-api"
Nov 03 21:32:32.044188 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.044157    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"
Nov 03 21:32:32.097430 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.097404    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-bb2b291b\""
Nov 03 21:32:32.143394 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.143362    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61123a59-6d54-4b06-9417-92de4371f28d-plugins-config-volume\") pod \"payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r\" (UID: \"61123a59-6d54-4b06-9417-92de4371f28d\") " pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"
Nov 03 21:32:32.143599 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.143566    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxp5z\" (UniqueName: \"kubernetes.io/projected/61123a59-6d54-4b06-9417-92de4371f28d-kube-api-access-nxp5z\") pod \"payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r\" (UID: \"61123a59-6d54-4b06-9417-92de4371f28d\") " pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"
Nov 03 21:32:32.149809 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.149781    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-bb2b291b-dockercfg-th9r4\""
Nov 03 21:32:32.151202 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.151180    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"]
Nov 03 21:32:32.211877 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.211850    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"]
Nov 03 21:32:32.215219 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.215201    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"
Nov 03 21:32:32.244401 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.244377    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a406b1a-fece-41ac-84dd-2804f2638419-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29\" (UID: \"5a406b1a-fece-41ac-84dd-2804f2638419\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"
Nov 03 21:32:32.244497 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.244405    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb758\" (UniqueName: \"kubernetes.io/projected/5a406b1a-fece-41ac-84dd-2804f2638419-kube-api-access-sb758\") pod \"payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29\" (UID: \"5a406b1a-fece-41ac-84dd-2804f2638419\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"
Nov 03 21:32:32.244559 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.244502    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxp5z\" (UniqueName: \"kubernetes.io/projected/61123a59-6d54-4b06-9417-92de4371f28d-kube-api-access-nxp5z\") pod \"payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r\" (UID: \"61123a59-6d54-4b06-9417-92de4371f28d\") " pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"
Nov 03 21:32:32.244559 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.244544    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61123a59-6d54-4b06-9417-92de4371f28d-plugins-config-volume\") pod \"payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r\" (UID: \"61123a59-6d54-4b06-9417-92de4371f28d\") " pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"
Nov 03 21:32:32.245171 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.245153    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61123a59-6d54-4b06-9417-92de4371f28d-plugins-config-volume\") pod \"payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r\" (UID: \"61123a59-6d54-4b06-9417-92de4371f28d\") " pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"
Nov 03 21:32:32.272367 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.272343    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"]
Nov 03 21:32:32.305733 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.305675    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxp5z\" (UniqueName: \"kubernetes.io/projected/61123a59-6d54-4b06-9417-92de4371f28d-kube-api-access-nxp5z\") pod \"payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r\" (UID: \"61123a59-6d54-4b06-9417-92de4371f28d\") " pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"
Nov 03 21:32:32.345537 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.345509    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a406b1a-fece-41ac-84dd-2804f2638419-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29\" (UID: \"5a406b1a-fece-41ac-84dd-2804f2638419\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"
Nov 03 21:32:32.345671 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.345546    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb758\" (UniqueName: \"kubernetes.io/projected/5a406b1a-fece-41ac-84dd-2804f2638419-kube-api-access-sb758\") pod \"payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29\" (UID: \"5a406b1a-fece-41ac-84dd-2804f2638419\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"
Nov 03 21:32:32.346145 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.346128    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a406b1a-fece-41ac-84dd-2804f2638419-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29\" (UID: \"5a406b1a-fece-41ac-84dd-2804f2638419\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"
Nov 03 21:32:32.358596 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.358569    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"
Nov 03 21:32:32.406960 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.406933    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb758\" (UniqueName: \"kubernetes.io/projected/5a406b1a-fece-41ac-84dd-2804f2638419-kube-api-access-sb758\") pod \"payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29\" (UID: \"5a406b1a-fece-41ac-84dd-2804f2638419\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"
Nov 03 21:32:32.528225 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.528193    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"
Nov 03 21:32:32.727736 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.727707    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"]
Nov 03 21:32:32.729604 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:32:32.729583    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61123a59_6d54_4b06_9417_92de4371f28d.slice/crio-0799f08b231b06f872a06d8c5991dbae3a9543a797abc90fc1c198b5acfbe857 WatchSource:0}: Error finding container 0799f08b231b06f872a06d8c5991dbae3a9543a797abc90fc1c198b5acfbe857: Status 404 returned error can't find the container with id 0799f08b231b06f872a06d8c5991dbae3a9543a797abc90fc1c198b5acfbe857
Nov 03 21:32:32.898386 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.898362    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"]
Nov 03 21:32:32.901104 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:32:32.901080    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a406b1a_fece_41ac_84dd_2804f2638419.slice/crio-ca7fc0b0f32dd1bb5e9289b380a62ae7336b4ac65682f310b9d7f75673e175a8 WatchSource:0}: Error finding container ca7fc0b0f32dd1bb5e9289b380a62ae7336b4ac65682f310b9d7f75673e175a8: Status 404 returned error can't find the container with id ca7fc0b0f32dd1bb5e9289b380a62ae7336b4ac65682f310b9d7f75673e175a8
Nov 03 21:32:32.923214 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.923192    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29" event={"ID":"5a406b1a-fece-41ac-84dd-2804f2638419","Type":"ContainerStarted","Data":"ca7fc0b0f32dd1bb5e9289b380a62ae7336b4ac65682f310b9d7f75673e175a8"}
Nov 03 21:32:32.924598 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.924574    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r" event={"ID":"61123a59-6d54-4b06-9417-92de4371f28d","Type":"ContainerStarted","Data":"e5283a5aee14383d72e1edd0d4c120e05152b36d3c31369dbf93fc923ca435ca"}
Nov 03 21:32:32.924733 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.924652    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r" event={"ID":"61123a59-6d54-4b06-9417-92de4371f28d","Type":"ContainerStarted","Data":"0799f08b231b06f872a06d8c5991dbae3a9543a797abc90fc1c198b5acfbe857"}
Nov 03 21:32:32.924733 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:32.924684    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"
Nov 03 21:32:33.041640 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.041575    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r" podStartSLOduration=2.04156145 podStartE2EDuration="2.04156145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:32:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:32:33.040330687 +0000 UTC m=+3155.585037059" watchObservedRunningTime="2026-11-03 21:32:33.04156145 +0000 UTC m=+3155.586267818"
Nov 03 21:32:33.269364 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.269330    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"]
Nov 03 21:32:33.272827 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.272813    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:33.324857 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.324835    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-bb2b291b-serving-cert\""
Nov 03 21:32:33.331680 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.331652    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"]
Nov 03 21:32:33.354836 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.354812    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-metrics-certs\") pod \"maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:33.354953 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.354872    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-maas-api-tls\") pod \"maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:33.355031 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.355011    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfsww\" (UniqueName: \"kubernetes.io/projected/960ea196-fa35-4a30-a862-f44ac9ebbe33-kube-api-access-jfsww\") pod \"maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:33.355075 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.355049    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/960ea196-fa35-4a30-a862-f44ac9ebbe33-openshift-service-ca\") pod \"maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:33.455769 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.455728    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfsww\" (UniqueName: \"kubernetes.io/projected/960ea196-fa35-4a30-a862-f44ac9ebbe33-kube-api-access-jfsww\") pod \"maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:33.455769 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.455770    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/960ea196-fa35-4a30-a862-f44ac9ebbe33-openshift-service-ca\") pod \"maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:33.456017 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.455826    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-metrics-certs\") pod \"maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:33.456017 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.455885    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-maas-api-tls\") pod \"maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:33.456017 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:32:33.456004    4993 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-serving-cert: secret "maas-api-e2e-ait-bb2b291b-serving-cert" not found
Nov 03 21:32:33.456173 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:32:33.456092    4993 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-maas-api-tls podName:960ea196-fa35-4a30-a862-f44ac9ebbe33 nodeName:}" failed. No retries permitted until 2026-11-03 21:32:33.956064759 +0000 UTC m=+3156.500771121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-maas-api-tls") pod "maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp" (UID: "960ea196-fa35-4a30-a862-f44ac9ebbe33") : secret "maas-api-e2e-ait-bb2b291b-serving-cert" not found
Nov 03 21:32:33.456512 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.456490    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/960ea196-fa35-4a30-a862-f44ac9ebbe33-openshift-service-ca\") pod \"maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:33.458269 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.458247    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-metrics-certs\") pod \"maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:33.517253 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.517217    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfsww\" (UniqueName: \"kubernetes.io/projected/960ea196-fa35-4a30-a862-f44ac9ebbe33-kube-api-access-jfsww\") pod \"maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:33.929941 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.929900    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29" event={"ID":"5a406b1a-fece-41ac-84dd-2804f2638419","Type":"ContainerStarted","Data":"221664f04b5084222dfc958a6b16995b1abf787627127c3f64825fad393a60ed"}
Nov 03 21:32:33.930375 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.930034    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"
Nov 03 21:32:33.960810 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.960771    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-maas-api-tls\") pod \"maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:33.963217 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:33.963191    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-maas-api-tls\") pod \"maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:34.046553 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:34.046501    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29" podStartSLOduration=2.046487291 podStartE2EDuration="2.046487291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:32:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:32:34.045134824 +0000 UTC m=+3156.589841194" watchObservedRunningTime="2026-11-03 21:32:34.046487291 +0000 UTC m=+3156.591193659"
Nov 03 21:32:34.185810 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:34.185731    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:34.561955 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:34.561929    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"]
Nov 03 21:32:34.564463 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:32:34.564434    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod960ea196_fa35_4a30_a862_f44ac9ebbe33.slice/crio-e54c0194717313156786620ab875495f7c33a1c706294c33358cfe267b4c35ea WatchSource:0}: Error finding container e54c0194717313156786620ab875495f7c33a1c706294c33358cfe267b4c35ea: Status 404 returned error can't find the container with id e54c0194717313156786620ab875495f7c33a1c706294c33358cfe267b4c35ea
Nov 03 21:32:34.936020 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:34.935901    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp" event={"ID":"960ea196-fa35-4a30-a862-f44ac9ebbe33","Type":"ContainerStarted","Data":"e54c0194717313156786620ab875495f7c33a1c706294c33358cfe267b4c35ea"}
Nov 03 21:32:35.942129 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:35.942085    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp" event={"ID":"960ea196-fa35-4a30-a862-f44ac9ebbe33","Type":"ContainerStarted","Data":"88494cf7ec5a8a327fda16e8a5e9c1bef60351536df7dc7bf523442aee3c9a2e"}
Nov 03 21:32:35.942548 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:35.942231    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:36.059915 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:36.059865    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp" podStartSLOduration=2.2275541739999998 podStartE2EDuration="3.059851354s" totalImagesPullingTime="832.29718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:32:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:32:36.058041972 +0000 UTC m=+3158.602748342" watchObservedRunningTime="2026-11-03 21:32:36.059851354 +0000 UTC m=+3158.604557723"
Nov 03 21:32:41.952302 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:41.952272    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:32:43.933578 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:43.933548    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"
Nov 03 21:32:44.939349 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:44.939318    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"
Nov 03 21:32:51.036459 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:51.036426    4993 generic.go:356] "Generic (PLEG): container finished" podID="deb43ff0-9d5c-4117-9d0b-324769219775" containerID="52e4608153d4cb3133dbd043b420be35899956ce0b343f2cd7e0d8a7b1c745c9" exitCode=0
Nov 03 21:32:51.036459 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:51.036471    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t" event={"ID":"deb43ff0-9d5c-4117-9d0b-324769219775","Type":"ContainerDied","Data":"52e4608153d4cb3133dbd043b420be35899956ce0b343f2cd7e0d8a7b1c745c9"}
Nov 03 21:32:51.036966 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:51.036818    4993 scope.go:122] "RemoveContainer" containerID="52e4608153d4cb3133dbd043b420be35899956ce0b343f2cd7e0d8a7b1c745c9"
Nov 03 21:32:51.037966 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:51.037938    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:32:51.038037 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:51.038010    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:32:51.038075 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:51.038043    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:32:52.043126 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:52.043095    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t" event={"ID":"deb43ff0-9d5c-4117-9d0b-324769219775","Type":"ContainerStarted","Data":"6bd669c5a6e3c7fcbb0441c36ec8997843c0cbbe938ae78da859e06948d7c585"}
Nov 03 21:32:53.019835 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:53.019796    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:53.021193 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:53.021166    4993 patch_prober.go:29] interesting pod/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 03 21:32:53.021317 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:53.021228    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t" podUID="deb43ff0-9d5c-4117-9d0b-324769219775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 03 21:32:54.020005 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:54.019972    4993 patch_prober.go:29] interesting pod/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 03 21:32:54.020397 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:54.020030    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t" podUID="deb43ff0-9d5c-4117-9d0b-324769219775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 03 21:32:55.019966 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:55.019931    4993 patch_prober.go:29] interesting pod/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 03 21:32:55.020149 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:55.019986    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t" podUID="deb43ff0-9d5c-4117-9d0b-324769219775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 03 21:32:56.019480 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:56.019449    4993 patch_prober.go:29] interesting pod/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 03 21:32:56.019666 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:56.019510    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t" podUID="deb43ff0-9d5c-4117-9d0b-324769219775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 03 21:32:56.896617 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:56.896580    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:57.019845 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:57.019811    4993 patch_prober.go:29] interesting pod/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Nov 03 21:32:57.020040 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:57.019877    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t" podUID="deb43ff0-9d5c-4117-9d0b-324769219775" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Nov 03 21:32:58.023645 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:58.023605    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:32:58.024564 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:32:58.024547    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:33:01.668979 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:33:01.668940    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:33:05.840220 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:05.840180    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"]
Nov 03 21:33:05.948609 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:05.948574    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"]
Nov 03 21:33:10.840713 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:10.840643    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r" podUID="61123a59-6d54-4b06-9417-92de4371f28d" containerName="payload-processing" containerID="cri-o://e5283a5aee14383d72e1edd0d4c120e05152b36d3c31369dbf93fc923ca435ca" gracePeriod=25
Nov 03 21:33:10.948959 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:10.948920    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29" podUID="5a406b1a-fece-41ac-84dd-2804f2638419" containerName="payload-pre-processing" containerID="cri-o://221664f04b5084222dfc958a6b16995b1abf787627127c3f64825fad393a60ed" gracePeriod=25
Nov 03 21:33:11.089730 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.089701    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"
Nov 03 21:33:11.139294 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.139255    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61123a59-6d54-4b06-9417-92de4371f28d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61123a59-6d54-4b06-9417-92de4371f28d-plugins-config-volume\") pod \"61123a59-6d54-4b06-9417-92de4371f28d\" (UID: \"61123a59-6d54-4b06-9417-92de4371f28d\") "
Nov 03 21:33:11.139294 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.139295    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61123a59-6d54-4b06-9417-92de4371f28d-kube-api-access-nxp5z\" (UniqueName: \"kubernetes.io/projected/61123a59-6d54-4b06-9417-92de4371f28d-kube-api-access-nxp5z\") pod \"61123a59-6d54-4b06-9417-92de4371f28d\" (UID: \"61123a59-6d54-4b06-9417-92de4371f28d\") "
Nov 03 21:33:11.139669 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.139645    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61123a59-6d54-4b06-9417-92de4371f28d-plugins-config-volume" pod "61123a59-6d54-4b06-9417-92de4371f28d" (UID: "61123a59-6d54-4b06-9417-92de4371f28d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:33:11.141598 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.141571    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61123a59-6d54-4b06-9417-92de4371f28d-kube-api-access-nxp5z" pod "61123a59-6d54-4b06-9417-92de4371f28d" (UID: "61123a59-6d54-4b06-9417-92de4371f28d"). InnerVolumeSpecName "kube-api-access-nxp5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:33:11.162078 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.162048    4993 generic.go:356] "Generic (PLEG): container finished" podID="5a406b1a-fece-41ac-84dd-2804f2638419" containerID="221664f04b5084222dfc958a6b16995b1abf787627127c3f64825fad393a60ed" exitCode=0
Nov 03 21:33:11.162171 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.162118    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29" event={"ID":"5a406b1a-fece-41ac-84dd-2804f2638419","Type":"ContainerDied","Data":"221664f04b5084222dfc958a6b16995b1abf787627127c3f64825fad393a60ed"}
Nov 03 21:33:11.163623 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.163597    4993 generic.go:356] "Generic (PLEG): container finished" podID="61123a59-6d54-4b06-9417-92de4371f28d" containerID="e5283a5aee14383d72e1edd0d4c120e05152b36d3c31369dbf93fc923ca435ca" exitCode=0
Nov 03 21:33:11.163728 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.163656    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r" event={"ID":"61123a59-6d54-4b06-9417-92de4371f28d","Type":"ContainerDied","Data":"e5283a5aee14383d72e1edd0d4c120e05152b36d3c31369dbf93fc923ca435ca"}
Nov 03 21:33:11.163728 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.163662    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"
Nov 03 21:33:11.163728 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.163705    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r" event={"ID":"61123a59-6d54-4b06-9417-92de4371f28d","Type":"ContainerDied","Data":"0799f08b231b06f872a06d8c5991dbae3a9543a797abc90fc1c198b5acfbe857"}
Nov 03 21:33:11.163840 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.163738    4993 scope.go:122] "RemoveContainer" containerID="e5283a5aee14383d72e1edd0d4c120e05152b36d3c31369dbf93fc923ca435ca"
Nov 03 21:33:11.173577 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.173557    4993 scope.go:122] "RemoveContainer" containerID="e5283a5aee14383d72e1edd0d4c120e05152b36d3c31369dbf93fc923ca435ca"
Nov 03 21:33:11.173860 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:33:11.173843    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5283a5aee14383d72e1edd0d4c120e05152b36d3c31369dbf93fc923ca435ca\": container with ID starting with e5283a5aee14383d72e1edd0d4c120e05152b36d3c31369dbf93fc923ca435ca not found: ID does not exist" containerID="e5283a5aee14383d72e1edd0d4c120e05152b36d3c31369dbf93fc923ca435ca"
Nov 03 21:33:11.173916 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.173867    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5283a5aee14383d72e1edd0d4c120e05152b36d3c31369dbf93fc923ca435ca"} err="failed to get container status \"e5283a5aee14383d72e1edd0d4c120e05152b36d3c31369dbf93fc923ca435ca\": rpc error: code = NotFound desc = could not find container \"e5283a5aee14383d72e1edd0d4c120e05152b36d3c31369dbf93fc923ca435ca\": container with ID starting with e5283a5aee14383d72e1edd0d4c120e05152b36d3c31369dbf93fc923ca435ca not found: ID does not exist"
Nov 03 21:33:11.187114 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.187095    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"
Nov 03 21:33:11.239932 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.239898    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a406b1a-fece-41ac-84dd-2804f2638419-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a406b1a-fece-41ac-84dd-2804f2638419-plugins-config-volume\") pod \"5a406b1a-fece-41ac-84dd-2804f2638419\" (UID: \"5a406b1a-fece-41ac-84dd-2804f2638419\") "
Nov 03 21:33:11.240113 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.240015    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a406b1a-fece-41ac-84dd-2804f2638419-kube-api-access-sb758\" (UniqueName: \"kubernetes.io/projected/5a406b1a-fece-41ac-84dd-2804f2638419-kube-api-access-sb758\") pod \"5a406b1a-fece-41ac-84dd-2804f2638419\" (UID: \"5a406b1a-fece-41ac-84dd-2804f2638419\") "
Nov 03 21:33:11.240255 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.240237    4993 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/61123a59-6d54-4b06-9417-92de4371f28d-plugins-config-volume\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:11.240326 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.240261    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxp5z\" (UniqueName: \"kubernetes.io/projected/61123a59-6d54-4b06-9417-92de4371f28d-kube-api-access-nxp5z\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:11.240376 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.240319    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a406b1a-fece-41ac-84dd-2804f2638419-plugins-config-volume" pod "5a406b1a-fece-41ac-84dd-2804f2638419" (UID: "5a406b1a-fece-41ac-84dd-2804f2638419"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:33:11.242171 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.242151    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a406b1a-fece-41ac-84dd-2804f2638419-kube-api-access-sb758" pod "5a406b1a-fece-41ac-84dd-2804f2638419" (UID: "5a406b1a-fece-41ac-84dd-2804f2638419"). InnerVolumeSpecName "kube-api-access-sb758". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:33:11.340909 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.340864    4993 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a406b1a-fece-41ac-84dd-2804f2638419-plugins-config-volume\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:11.340909 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.340895    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sb758\" (UniqueName: \"kubernetes.io/projected/5a406b1a-fece-41ac-84dd-2804f2638419-kube-api-access-sb758\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:11.430387 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.430305    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"]
Nov 03 21:33:11.434220 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.434194    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-bb2b291b-6fc94d6774-k7t2r"]
Nov 03 21:33:11.575359 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:11.575326    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61123a59-6d54-4b06-9417-92de4371f28d" path="/var/lib/kubelet/pods/61123a59-6d54-4b06-9417-92de4371f28d/volumes"
Nov 03 21:33:12.151078 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.150997    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"]
Nov 03 21:33:12.151524 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.151199    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp" podUID="960ea196-fa35-4a30-a862-f44ac9ebbe33" containerName="maas-api" containerID="cri-o://88494cf7ec5a8a327fda16e8a5e9c1bef60351536df7dc7bf523442aee3c9a2e" gracePeriod=30
Nov 03 21:33:12.172006 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.171971    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29" event={"ID":"5a406b1a-fece-41ac-84dd-2804f2638419","Type":"ContainerDied","Data":"ca7fc0b0f32dd1bb5e9289b380a62ae7336b4ac65682f310b9d7f75673e175a8"}
Nov 03 21:33:12.172006 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.172003    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"
Nov 03 21:33:12.172154 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.172056    4993 scope.go:122] "RemoveContainer" containerID="221664f04b5084222dfc958a6b16995b1abf787627127c3f64825fad393a60ed"
Nov 03 21:33:12.380605 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.380563    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"]
Nov 03 21:33:12.386152 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.386115    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-bb2b291b-54996dfdf5-tdx29"]
Nov 03 21:33:12.414052 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.413999    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:33:12.450263 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.450239    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/960ea196-fa35-4a30-a862-f44ac9ebbe33-kube-api-access-jfsww\" (UniqueName: \"kubernetes.io/projected/960ea196-fa35-4a30-a862-f44ac9ebbe33-kube-api-access-jfsww\") pod \"960ea196-fa35-4a30-a862-f44ac9ebbe33\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") "
Nov 03 21:33:12.450418 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.450279    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/960ea196-fa35-4a30-a862-f44ac9ebbe33-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/960ea196-fa35-4a30-a862-f44ac9ebbe33-openshift-service-ca\") pod \"960ea196-fa35-4a30-a862-f44ac9ebbe33\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") "
Nov 03 21:33:12.450494 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.450411    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-metrics-certs\" (UniqueName: \"kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-metrics-certs\") pod \"960ea196-fa35-4a30-a862-f44ac9ebbe33\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") "
Nov 03 21:33:12.450494 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.450459    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-maas-api-tls\") pod \"960ea196-fa35-4a30-a862-f44ac9ebbe33\" (UID: \"960ea196-fa35-4a30-a862-f44ac9ebbe33\") "
Nov 03 21:33:12.450646 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.450613    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/960ea196-fa35-4a30-a862-f44ac9ebbe33-openshift-service-ca" pod "960ea196-fa35-4a30-a862-f44ac9ebbe33" (UID: "960ea196-fa35-4a30-a862-f44ac9ebbe33"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:33:12.450852 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.450828    4993 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/960ea196-fa35-4a30-a862-f44ac9ebbe33-openshift-service-ca\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:12.452593 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.452570    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-maas-api-tls" pod "960ea196-fa35-4a30-a862-f44ac9ebbe33" (UID: "960ea196-fa35-4a30-a862-f44ac9ebbe33"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:33:12.452698 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.452610    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-metrics-certs" pod "960ea196-fa35-4a30-a862-f44ac9ebbe33" (UID: "960ea196-fa35-4a30-a862-f44ac9ebbe33"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:33:12.452698 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.452645    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/960ea196-fa35-4a30-a862-f44ac9ebbe33-kube-api-access-jfsww" pod "960ea196-fa35-4a30-a862-f44ac9ebbe33" (UID: "960ea196-fa35-4a30-a862-f44ac9ebbe33"). InnerVolumeSpecName "kube-api-access-jfsww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:33:12.551814 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.551774    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfsww\" (UniqueName: \"kubernetes.io/projected/960ea196-fa35-4a30-a862-f44ac9ebbe33-kube-api-access-jfsww\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:12.551814 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.551816    4993 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-metrics-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:12.552077 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:12.551833    4993 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/960ea196-fa35-4a30-a862-f44ac9ebbe33-maas-api-tls\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:13.178476 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:13.178441    4993 generic.go:356] "Generic (PLEG): container finished" podID="960ea196-fa35-4a30-a862-f44ac9ebbe33" containerID="88494cf7ec5a8a327fda16e8a5e9c1bef60351536df7dc7bf523442aee3c9a2e" exitCode=0
Nov 03 21:33:13.178934 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:13.178485    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp" event={"ID":"960ea196-fa35-4a30-a862-f44ac9ebbe33","Type":"ContainerDied","Data":"88494cf7ec5a8a327fda16e8a5e9c1bef60351536df7dc7bf523442aee3c9a2e"}
Nov 03 21:33:13.178934 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:13.178492    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"
Nov 03 21:33:13.178934 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:13.178524    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp" event={"ID":"960ea196-fa35-4a30-a862-f44ac9ebbe33","Type":"ContainerDied","Data":"e54c0194717313156786620ab875495f7c33a1c706294c33358cfe267b4c35ea"}
Nov 03 21:33:13.178934 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:13.178546    4993 scope.go:122] "RemoveContainer" containerID="88494cf7ec5a8a327fda16e8a5e9c1bef60351536df7dc7bf523442aee3c9a2e"
Nov 03 21:33:13.187449 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:13.187434    4993 scope.go:122] "RemoveContainer" containerID="88494cf7ec5a8a327fda16e8a5e9c1bef60351536df7dc7bf523442aee3c9a2e"
Nov 03 21:33:13.187757 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:33:13.187734    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88494cf7ec5a8a327fda16e8a5e9c1bef60351536df7dc7bf523442aee3c9a2e\": container with ID starting with 88494cf7ec5a8a327fda16e8a5e9c1bef60351536df7dc7bf523442aee3c9a2e not found: ID does not exist" containerID="88494cf7ec5a8a327fda16e8a5e9c1bef60351536df7dc7bf523442aee3c9a2e"
Nov 03 21:33:13.187826 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:13.187767    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88494cf7ec5a8a327fda16e8a5e9c1bef60351536df7dc7bf523442aee3c9a2e"} err="failed to get container status \"88494cf7ec5a8a327fda16e8a5e9c1bef60351536df7dc7bf523442aee3c9a2e\": rpc error: code = NotFound desc = could not find container \"88494cf7ec5a8a327fda16e8a5e9c1bef60351536df7dc7bf523442aee3c9a2e\": container with ID starting with 88494cf7ec5a8a327fda16e8a5e9c1bef60351536df7dc7bf523442aee3c9a2e not found: ID does not exist"
Nov 03 21:33:13.340754 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:13.340718    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"]
Nov 03 21:33:13.344829 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:13.344796    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bb2b291b-794fb94d55-5fbsp"]
Nov 03 21:33:13.576499 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:13.576468    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a406b1a-fece-41ac-84dd-2804f2638419" path="/var/lib/kubelet/pods/5a406b1a-fece-41ac-84dd-2804f2638419/volumes"
Nov 03 21:33:13.576889 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:13.576876    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="960ea196-fa35-4a30-a862-f44ac9ebbe33" path="/var/lib/kubelet/pods/960ea196-fa35-4a30-a862-f44ac9ebbe33/volumes"
Nov 03 21:33:15.322501 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:15.322470    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"]
Nov 03 21:33:15.322970 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:15.322717    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t" podUID="deb43ff0-9d5c-4117-9d0b-324769219775" containerName="istio-proxy" containerID="cri-o://6bd669c5a6e3c7fcbb0441c36ec8997843c0cbbe938ae78da859e06948d7c585" gracePeriod=30
Nov 03 21:33:17.041385 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.041340    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"]
Nov 03 21:33:17.041957 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.041935    4993 cpu_manager.go:414] "containerMap: removing container" podUID="61123a59-6d54-4b06-9417-92de4371f28d" containerName="payload-processing"
Nov 03 21:33:17.042058 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.041962    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61123a59-6d54-4b06-9417-92de4371f28d" containerName="payload-processing"
Nov 03 21:33:17.042058 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.041978    4993 cpu_manager.go:414] "containerMap: removing container" podUID="5a406b1a-fece-41ac-84dd-2804f2638419" containerName="payload-pre-processing"
Nov 03 21:33:17.042058 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.041986    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a406b1a-fece-41ac-84dd-2804f2638419" containerName="payload-pre-processing"
Nov 03 21:33:17.042058 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.042034    4993 cpu_manager.go:414] "containerMap: removing container" podUID="960ea196-fa35-4a30-a862-f44ac9ebbe33" containerName="maas-api"
Nov 03 21:33:17.042058 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.042042    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="960ea196-fa35-4a30-a862-f44ac9ebbe33" containerName="maas-api"
Nov 03 21:33:17.042305 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.042165    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="960ea196-fa35-4a30-a862-f44ac9ebbe33" containerName="maas-api"
Nov 03 21:33:17.042305 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.042184    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="61123a59-6d54-4b06-9417-92de4371f28d" containerName="payload-processing"
Nov 03 21:33:17.042305 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.042193    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a406b1a-fece-41ac-84dd-2804f2638419" containerName="payload-pre-processing"
Nov 03 21:33:17.046852 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.046828    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.095980 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.095947    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-workload-certs\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.096174 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.096003    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istiod-ca-cert\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.096174 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.096096    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-podinfo\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.096270 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.096226    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-workload-socket\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.096315 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.096290    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-credential-socket\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.096367 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.096319    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-envoy\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.096475 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.096456    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-token\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.096533 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.096495    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42x6k\" (UniqueName: \"kubernetes.io/projected/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-kube-api-access-42x6k\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.096584 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.096548    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-data\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.105598 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.105577    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-authcfg-http-220932-openshift-default-dockercfg-p6cm7\""
Nov 03 21:33:17.155781 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.155749    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"]
Nov 03 21:33:17.197826 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.197784    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-workload-certs\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.198020 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.197841    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istiod-ca-cert\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.198020 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.197869    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-podinfo\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.198020 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.197922    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-workload-socket\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.198020 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.197963    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-credential-socket\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.198020 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.197987    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-envoy\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.198020 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.198017    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-token\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.198341 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.198045    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42x6k\" (UniqueName: \"kubernetes.io/projected/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-kube-api-access-42x6k\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.198341 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.198084    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-data\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.198456 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.198351    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-credential-socket\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.198456 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.198409    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-data\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.198954 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.198618    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-workload-socket\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.199375 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.199354    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istiod-ca-cert\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.199561 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.199544    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-workload-certs\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.200738 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.200715    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-envoy\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.201182 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.201166    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-podinfo\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.259345 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.259310    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-token\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.259789 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.259762    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42x6k\" (UniqueName: \"kubernetes.io/projected/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-kube-api-access-42x6k\") pod \"e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") " pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.359759 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.359679    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:17.786017 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.785993    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"]
Nov 03 21:33:17.788538 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:33:17.788514    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee92fab9_aa65_40ea_9c8f_7cd7f2142e02.slice/crio-c92869840a3dd0d79f91f96a466a1ec1dc315eb0bb14532c0526f49b7eea4afe WatchSource:0}: Error finding container c92869840a3dd0d79f91f96a466a1ec1dc315eb0bb14532c0526f49b7eea4afe: Status 404 returned error can't find the container with id c92869840a3dd0d79f91f96a466a1ec1dc315eb0bb14532c0526f49b7eea4afe
Nov 03 21:33:17.790788 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.790758    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:33:17.790863 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.790824    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:33:17.790907 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:17.790866    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:33:18.204654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:18.204552    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs" event={"ID":"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02","Type":"ContainerStarted","Data":"64935c8a056d5ff2e185811bdbaf1094310400a8006a1856ae290198c74730c2"}
Nov 03 21:33:18.204654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:18.204614    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs" event={"ID":"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02","Type":"ContainerStarted","Data":"c92869840a3dd0d79f91f96a466a1ec1dc315eb0bb14532c0526f49b7eea4afe"}
Nov 03 21:33:18.325693 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:18.325614    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs" podStartSLOduration=2.325599166 podStartE2EDuration="2.325599166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:33:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:33:18.323720805 +0000 UTC m=+3200.868427176" watchObservedRunningTime="2026-11-03 21:33:18.325599166 +0000 UTC m=+3200.870305535"
Nov 03 21:33:18.360870 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:18.360831    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:18.365940 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:18.365911    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:19.209359 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:19.209326    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:19.210300 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:19.210281    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:20.573597 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.573573    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:33:20.632015 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.631942    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-istio-envoy\") pod \"deb43ff0-9d5c-4117-9d0b-324769219775\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") "
Nov 03 21:33:20.632015 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.631978    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/deb43ff0-9d5c-4117-9d0b-324769219775-istio-token\" (UniqueName: \"kubernetes.io/projected/deb43ff0-9d5c-4117-9d0b-324769219775-istio-token\") pod \"deb43ff0-9d5c-4117-9d0b-324769219775\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") "
Nov 03 21:33:20.632015 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632013    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-credential-socket\") pod \"deb43ff0-9d5c-4117-9d0b-324769219775\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") "
Nov 03 21:33:20.632275 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632047    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-istio-data\") pod \"deb43ff0-9d5c-4117-9d0b-324769219775\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") "
Nov 03 21:33:20.632275 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632088    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/deb43ff0-9d5c-4117-9d0b-324769219775-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/deb43ff0-9d5c-4117-9d0b-324769219775-istiod-ca-cert\") pod \"deb43ff0-9d5c-4117-9d0b-324769219775\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") "
Nov 03 21:33:20.632275 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632122    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-workload-certs\") pod \"deb43ff0-9d5c-4117-9d0b-324769219775\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") "
Nov 03 21:33:20.632275 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632187    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-workload-socket\") pod \"deb43ff0-9d5c-4117-9d0b-324769219775\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") "
Nov 03 21:33:20.632275 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632216    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/deb43ff0-9d5c-4117-9d0b-324769219775-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/deb43ff0-9d5c-4117-9d0b-324769219775-istio-podinfo\") pod \"deb43ff0-9d5c-4117-9d0b-324769219775\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") "
Nov 03 21:33:20.632532 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632325    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/deb43ff0-9d5c-4117-9d0b-324769219775-kube-api-access-jbfbb\" (UniqueName: \"kubernetes.io/projected/deb43ff0-9d5c-4117-9d0b-324769219775-kube-api-access-jbfbb\") pod \"deb43ff0-9d5c-4117-9d0b-324769219775\" (UID: \"deb43ff0-9d5c-4117-9d0b-324769219775\") "
Nov 03 21:33:20.632532 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632369    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-istio-data" pod "deb43ff0-9d5c-4117-9d0b-324769219775" (UID: "deb43ff0-9d5c-4117-9d0b-324769219775"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:33:20.632532 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632373    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-credential-socket" pod "deb43ff0-9d5c-4117-9d0b-324769219775" (UID: "deb43ff0-9d5c-4117-9d0b-324769219775"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:33:20.632532 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632497    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-workload-socket" pod "deb43ff0-9d5c-4117-9d0b-324769219775" (UID: "deb43ff0-9d5c-4117-9d0b-324769219775"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:33:20.632782 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632538    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/deb43ff0-9d5c-4117-9d0b-324769219775-istiod-ca-cert" pod "deb43ff0-9d5c-4117-9d0b-324769219775" (UID: "deb43ff0-9d5c-4117-9d0b-324769219775"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:33:20.632782 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632749    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-workload-certs" pod "deb43ff0-9d5c-4117-9d0b-324769219775" (UID: "deb43ff0-9d5c-4117-9d0b-324769219775"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:33:20.633037 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632895    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:20.633037 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632911    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:20.633037 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632923    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/deb43ff0-9d5c-4117-9d0b-324769219775-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:20.633037 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632937    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:20.633037 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.632950    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:20.634456 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.634434    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-istio-envoy" pod "deb43ff0-9d5c-4117-9d0b-324769219775" (UID: "deb43ff0-9d5c-4117-9d0b-324769219775"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:33:20.634549 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.634528    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/deb43ff0-9d5c-4117-9d0b-324769219775-istio-podinfo" pod "deb43ff0-9d5c-4117-9d0b-324769219775" (UID: "deb43ff0-9d5c-4117-9d0b-324769219775"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:33:20.634953 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.634926    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deb43ff0-9d5c-4117-9d0b-324769219775-kube-api-access-jbfbb" pod "deb43ff0-9d5c-4117-9d0b-324769219775" (UID: "deb43ff0-9d5c-4117-9d0b-324769219775"). InnerVolumeSpecName "kube-api-access-jbfbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:33:20.635046 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.634926    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/deb43ff0-9d5c-4117-9d0b-324769219775-istio-token" pod "deb43ff0-9d5c-4117-9d0b-324769219775" (UID: "deb43ff0-9d5c-4117-9d0b-324769219775"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:33:20.734024 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.733992    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/deb43ff0-9d5c-4117-9d0b-324769219775-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:20.734024 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.734019    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/deb43ff0-9d5c-4117-9d0b-324769219775-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:20.734024 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.734029    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbfbb\" (UniqueName: \"kubernetes.io/projected/deb43ff0-9d5c-4117-9d0b-324769219775-kube-api-access-jbfbb\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:20.734270 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:20.734041    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/deb43ff0-9d5c-4117-9d0b-324769219775-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:33:21.219497 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:21.219457    4993 generic.go:356] "Generic (PLEG): container finished" podID="deb43ff0-9d5c-4117-9d0b-324769219775" containerID="6bd669c5a6e3c7fcbb0441c36ec8997843c0cbbe938ae78da859e06948d7c585" exitCode=0
Nov 03 21:33:21.219665 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:21.219512    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"
Nov 03 21:33:21.219665 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:21.219542    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t" event={"ID":"deb43ff0-9d5c-4117-9d0b-324769219775","Type":"ContainerDied","Data":"6bd669c5a6e3c7fcbb0441c36ec8997843c0cbbe938ae78da859e06948d7c585"}
Nov 03 21:33:21.219665 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:21.219591    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t" event={"ID":"deb43ff0-9d5c-4117-9d0b-324769219775","Type":"ContainerDied","Data":"53dc90fbe3bb935fc09349f2bd1b561598a330b0f83f74321b4618726e4d0d3a"}
Nov 03 21:33:21.219665 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:21.219612    4993 scope.go:122] "RemoveContainer" containerID="6bd669c5a6e3c7fcbb0441c36ec8997843c0cbbe938ae78da859e06948d7c585"
Nov 03 21:33:21.229366 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:21.229351    4993 scope.go:122] "RemoveContainer" containerID="52e4608153d4cb3133dbd043b420be35899956ce0b343f2cd7e0d8a7b1c745c9"
Nov 03 21:33:21.237436 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:21.237418    4993 scope.go:122] "RemoveContainer" containerID="6bd669c5a6e3c7fcbb0441c36ec8997843c0cbbe938ae78da859e06948d7c585"
Nov 03 21:33:21.237724 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:33:21.237707    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bd669c5a6e3c7fcbb0441c36ec8997843c0cbbe938ae78da859e06948d7c585\": container with ID starting with 6bd669c5a6e3c7fcbb0441c36ec8997843c0cbbe938ae78da859e06948d7c585 not found: ID does not exist" containerID="6bd669c5a6e3c7fcbb0441c36ec8997843c0cbbe938ae78da859e06948d7c585"
Nov 03 21:33:21.237767 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:21.237734    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bd669c5a6e3c7fcbb0441c36ec8997843c0cbbe938ae78da859e06948d7c585"} err="failed to get container status \"6bd669c5a6e3c7fcbb0441c36ec8997843c0cbbe938ae78da859e06948d7c585\": rpc error: code = NotFound desc = could not find container \"6bd669c5a6e3c7fcbb0441c36ec8997843c0cbbe938ae78da859e06948d7c585\": container with ID starting with 6bd669c5a6e3c7fcbb0441c36ec8997843c0cbbe938ae78da859e06948d7c585 not found: ID does not exist"
Nov 03 21:33:21.237767 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:21.237752    4993 scope.go:122] "RemoveContainer" containerID="52e4608153d4cb3133dbd043b420be35899956ce0b343f2cd7e0d8a7b1c745c9"
Nov 03 21:33:21.237998 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:33:21.237981    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52e4608153d4cb3133dbd043b420be35899956ce0b343f2cd7e0d8a7b1c745c9\": container with ID starting with 52e4608153d4cb3133dbd043b420be35899956ce0b343f2cd7e0d8a7b1c745c9 not found: ID does not exist" containerID="52e4608153d4cb3133dbd043b420be35899956ce0b343f2cd7e0d8a7b1c745c9"
Nov 03 21:33:21.238036 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:21.238006    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52e4608153d4cb3133dbd043b420be35899956ce0b343f2cd7e0d8a7b1c745c9"} err="failed to get container status \"52e4608153d4cb3133dbd043b420be35899956ce0b343f2cd7e0d8a7b1c745c9\": rpc error: code = NotFound desc = could not find container \"52e4608153d4cb3133dbd043b420be35899956ce0b343f2cd7e0d8a7b1c745c9\": container with ID starting with 52e4608153d4cb3133dbd043b420be35899956ce0b343f2cd7e0d8a7b1c745c9 not found: ID does not exist"
Nov 03 21:33:21.391154 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:21.391119    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"]
Nov 03 21:33:21.440571 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:21.440536    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-bb2b291b-openshift-default-68d58bc8f-w8p8t"]
Nov 03 21:33:21.576960 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:21.576891    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="deb43ff0-9d5c-4117-9d0b-324769219775" path="/var/lib/kubelet/pods/deb43ff0-9d5c-4117-9d0b-324769219775/volumes"
Nov 03 21:33:23.578152 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.578116    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"]
Nov 03 21:33:23.578510 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.578484    4993 cpu_manager.go:414] "containerMap: removing container" podUID="deb43ff0-9d5c-4117-9d0b-324769219775" containerName="istio-proxy"
Nov 03 21:33:23.578510 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.578495    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="deb43ff0-9d5c-4117-9d0b-324769219775" containerName="istio-proxy"
Nov 03 21:33:23.578510 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.578507    4993 cpu_manager.go:414] "containerMap: removing container" podUID="deb43ff0-9d5c-4117-9d0b-324769219775" containerName="istio-proxy"
Nov 03 21:33:23.578864 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.578512    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="deb43ff0-9d5c-4117-9d0b-324769219775" containerName="istio-proxy"
Nov 03 21:33:23.578864 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.578560    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="deb43ff0-9d5c-4117-9d0b-324769219775" containerName="istio-proxy"
Nov 03 21:33:23.578864 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.578567    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="deb43ff0-9d5c-4117-9d0b-324769219775" containerName="istio-proxy"
Nov 03 21:33:23.582691 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.582676    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:23.661033 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.661002    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:23.661216 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.661039    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-metrics-certs\") pod \"maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:23.661283 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.661221    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chmjw\" (UniqueName: \"kubernetes.io/projected/6a9e8850-b0b9-4904-86d1-274359e5af0f-kube-api-access-chmjw\") pod \"maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:23.661342 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.661263    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9e8850-b0b9-4904-86d1-274359e5af0f-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:23.684206 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.684179    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-authcfg-http-220932-serving-cert\""
Nov 03 21:33:23.685619 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.685597    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"]
Nov 03 21:33:23.762777 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.762745    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chmjw\" (UniqueName: \"kubernetes.io/projected/6a9e8850-b0b9-4904-86d1-274359e5af0f-kube-api-access-chmjw\") pod \"maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:23.762969 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.762781    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9e8850-b0b9-4904-86d1-274359e5af0f-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:23.762969 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.762805    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:23.762969 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.762835    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-metrics-certs\") pod \"maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:23.762969 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:33:23.762908    4993 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-serving-cert: secret "maas-api-e2e-authcfg-http-220932-serving-cert" not found
Nov 03 21:33:23.763195 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:33:23.762980    4993 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-maas-api-tls podName:6a9e8850-b0b9-4904-86d1-274359e5af0f nodeName:}" failed. No retries permitted until 2026-11-03 21:33:24.262964397 +0000 UTC m=+3206.807670767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-maas-api-tls") pod "maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd" (UID: "6a9e8850-b0b9-4904-86d1-274359e5af0f") : secret "maas-api-e2e-authcfg-http-220932-serving-cert" not found
Nov 03 21:33:23.763507 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.763489    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9e8850-b0b9-4904-86d1-274359e5af0f-openshift-service-ca\") pod \"maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:23.765359 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.765340    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-metrics-certs\") pod \"maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:23.823963 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:23.823926    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chmjw\" (UniqueName: \"kubernetes.io/projected/6a9e8850-b0b9-4904-86d1-274359e5af0f-kube-api-access-chmjw\") pod \"maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:24.267608 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:24.267564    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:24.270516 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:24.270487    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-maas-api-tls\") pod \"maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:24.496322 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:24.496288    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:24.868590 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:24.868539    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"]
Nov 03 21:33:24.871349 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:33:24.871323    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a9e8850_b0b9_4904_86d1_274359e5af0f.slice/crio-32fee8486d7cd32df31aeec6a111011be099784ed3d044b70be3257ca2a2dc15 WatchSource:0}: Error finding container 32fee8486d7cd32df31aeec6a111011be099784ed3d044b70be3257ca2a2dc15: Status 404 returned error can't find the container with id 32fee8486d7cd32df31aeec6a111011be099784ed3d044b70be3257ca2a2dc15
Nov 03 21:33:24.903306 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:24.903271    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"]
Nov 03 21:33:24.907435 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:24.907405    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"
Nov 03 21:33:24.959551 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:24.959516    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-authcfg-http-220932\""
Nov 03 21:33:24.959792 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:24.959775    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-authcfg-http-220932-dockercfg-sg6xc\""
Nov 03 21:33:24.974218 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:24.974187    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvqq7\" (UniqueName: \"kubernetes.io/projected/f20f2047-7442-4403-882e-1f51302d4546-kube-api-access-xvqq7\") pod \"payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq\" (UID: \"f20f2047-7442-4403-882e-1f51302d4546\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"
Nov 03 21:33:24.974343 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:24.974299    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f20f2047-7442-4403-882e-1f51302d4546-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq\" (UID: \"f20f2047-7442-4403-882e-1f51302d4546\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"
Nov 03 21:33:24.979570 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:24.979541    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"]
Nov 03 21:33:25.075621 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.075579    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvqq7\" (UniqueName: \"kubernetes.io/projected/f20f2047-7442-4403-882e-1f51302d4546-kube-api-access-xvqq7\") pod \"payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq\" (UID: \"f20f2047-7442-4403-882e-1f51302d4546\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"
Nov 03 21:33:25.075812 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.075678    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f20f2047-7442-4403-882e-1f51302d4546-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq\" (UID: \"f20f2047-7442-4403-882e-1f51302d4546\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"
Nov 03 21:33:25.076288 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.076263    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"]
Nov 03 21:33:25.076436 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.076413    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f20f2047-7442-4403-882e-1f51302d4546-plugins-config-volume\") pod \"payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq\" (UID: \"f20f2047-7442-4403-882e-1f51302d4546\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"
Nov 03 21:33:25.079589 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.079571    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"
Nov 03 21:33:25.137061 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.136984    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvqq7\" (UniqueName: \"kubernetes.io/projected/f20f2047-7442-4403-882e-1f51302d4546-kube-api-access-xvqq7\") pod \"payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq\" (UID: \"f20f2047-7442-4403-882e-1f51302d4546\") " pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"
Nov 03 21:33:25.137210 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.137182    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"]
Nov 03 21:33:25.176893 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.176863    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v259j\" (UniqueName: \"kubernetes.io/projected/8d159865-b910-40dc-b8a8-8c765ffbaf5e-kube-api-access-v259j\") pod \"payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k\" (UID: \"8d159865-b910-40dc-b8a8-8c765ffbaf5e\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"
Nov 03 21:33:25.177028 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.176934    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d159865-b910-40dc-b8a8-8c765ffbaf5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k\" (UID: \"8d159865-b910-40dc-b8a8-8c765ffbaf5e\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"
Nov 03 21:33:25.223242 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.223219    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"
Nov 03 21:33:25.240123 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.240098    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd" event={"ID":"6a9e8850-b0b9-4904-86d1-274359e5af0f","Type":"ContainerStarted","Data":"32fee8486d7cd32df31aeec6a111011be099784ed3d044b70be3257ca2a2dc15"}
Nov 03 21:33:25.278261 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.278226    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d159865-b910-40dc-b8a8-8c765ffbaf5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k\" (UID: \"8d159865-b910-40dc-b8a8-8c765ffbaf5e\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"
Nov 03 21:33:25.278531 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.278511    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v259j\" (UniqueName: \"kubernetes.io/projected/8d159865-b910-40dc-b8a8-8c765ffbaf5e-kube-api-access-v259j\") pod \"payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k\" (UID: \"8d159865-b910-40dc-b8a8-8c765ffbaf5e\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"
Nov 03 21:33:25.279532 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.279075    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d159865-b910-40dc-b8a8-8c765ffbaf5e-plugins-config-volume\") pod \"payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k\" (UID: \"8d159865-b910-40dc-b8a8-8c765ffbaf5e\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"
Nov 03 21:33:25.339600 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.339572    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v259j\" (UniqueName: \"kubernetes.io/projected/8d159865-b910-40dc-b8a8-8c765ffbaf5e-kube-api-access-v259j\") pod \"payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k\" (UID: \"8d159865-b910-40dc-b8a8-8c765ffbaf5e\") " pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"
Nov 03 21:33:25.393014 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.392953    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"
Nov 03 21:33:25.593579 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.593559    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"]
Nov 03 21:33:25.595520 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:33:25.595492    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf20f2047_7442_4403_882e_1f51302d4546.slice/crio-f96d7e888950b6eea20e225476c89e02b9faca77268259a35112e4e52d08e4a3 WatchSource:0}: Error finding container f96d7e888950b6eea20e225476c89e02b9faca77268259a35112e4e52d08e4a3: Status 404 returned error can't find the container with id f96d7e888950b6eea20e225476c89e02b9faca77268259a35112e4e52d08e4a3
Nov 03 21:33:25.764039 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:25.763867    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"]
Nov 03 21:33:25.767550 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:33:25.767516    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d159865_b910_40dc_b8a8_8c765ffbaf5e.slice/crio-bdea6e413fe37c72528912ce488b018f949dc4e50e909ca4ae9f9b311d146c83 WatchSource:0}: Error finding container bdea6e413fe37c72528912ce488b018f949dc4e50e909ca4ae9f9b311d146c83: Status 404 returned error can't find the container with id bdea6e413fe37c72528912ce488b018f949dc4e50e909ca4ae9f9b311d146c83
Nov 03 21:33:26.247778 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:26.247730    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k" event={"ID":"8d159865-b910-40dc-b8a8-8c765ffbaf5e","Type":"ContainerStarted","Data":"7b2ec461c02352e14c539607c1544d4f334f22eba43980b27625a27b8fd4395d"}
Nov 03 21:33:26.248247 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:26.247801    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k" event={"ID":"8d159865-b910-40dc-b8a8-8c765ffbaf5e","Type":"ContainerStarted","Data":"bdea6e413fe37c72528912ce488b018f949dc4e50e909ca4ae9f9b311d146c83"}
Nov 03 21:33:26.248247 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:26.247832    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"
Nov 03 21:33:26.249496 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:26.249472    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd" event={"ID":"6a9e8850-b0b9-4904-86d1-274359e5af0f","Type":"ContainerStarted","Data":"d9a1fd84f45b6e749efd00a788fb576bf9b3e4aeab583fbe9bb0513da2efaa55"}
Nov 03 21:33:26.249659 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:26.249516    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:26.251220 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:26.251192    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq" event={"ID":"f20f2047-7442-4403-882e-1f51302d4546","Type":"ContainerStarted","Data":"8dafc9f1c0b429841b4cb507abf79bd594c61971904c59b06c86a80eb7ca2de5"}
Nov 03 21:33:26.251358 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:26.251244    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq" event={"ID":"f20f2047-7442-4403-882e-1f51302d4546","Type":"ContainerStarted","Data":"f96d7e888950b6eea20e225476c89e02b9faca77268259a35112e4e52d08e4a3"}
Nov 03 21:33:26.251358 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:26.251265    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"
Nov 03 21:33:26.365582 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:26.365529    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k" podStartSLOduration=1.365513052 podStartE2EDuration="1.365513052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:33:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:33:26.364442918 +0000 UTC m=+3208.909149288" watchObservedRunningTime="2026-11-03 21:33:26.365513052 +0000 UTC m=+3208.910219420"
Nov 03 21:33:26.481662 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:26.481574    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd" podStartSLOduration=2.6839382560000002 podStartE2EDuration="3.481560349s" totalImagesPullingTime="797.622093ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:33:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:33:26.480484052 +0000 UTC m=+3209.025190416" watchObservedRunningTime="2026-11-03 21:33:26.481560349 +0000 UTC m=+3209.026266719"
Nov 03 21:33:26.595602 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:26.595494    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq" podStartSLOduration=2.595472936 podStartE2EDuration="2.595472936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:33:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:33:26.59482441 +0000 UTC m=+3209.139530773" watchObservedRunningTime="2026-11-03 21:33:26.595472936 +0000 UTC m=+3209.140179298"
Nov 03 21:33:31.693662 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:33:31.693607    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:33:32.262651 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:32.262608    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:33:37.260149 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:37.260121    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"
Nov 03 21:33:37.260574 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:37.260189    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"
Nov 03 21:33:39.333701 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:39.333667    4993 generic.go:356] "Generic (PLEG): container finished" podID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" containerID="64935c8a056d5ff2e185811bdbaf1094310400a8006a1856ae290198c74730c2" exitCode=0
Nov 03 21:33:39.334188 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:39.333744    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs" event={"ID":"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02","Type":"ContainerDied","Data":"64935c8a056d5ff2e185811bdbaf1094310400a8006a1856ae290198c74730c2"}
Nov 03 21:33:39.334188 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:39.334073    4993 scope.go:122] "RemoveContainer" containerID="64935c8a056d5ff2e185811bdbaf1094310400a8006a1856ae290198c74730c2"
Nov 03 21:33:39.335198 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:39.335173    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:33:39.335252 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:39.335240    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:33:39.335292 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:39.335272    4993 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Nov 03 21:33:40.340571 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:40.340537    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs" event={"ID":"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02","Type":"ContainerStarted","Data":"f27225b6aefd29dcd99fb753ee7f498be0a271b463e4bf93189d474c22ab4009"}
Nov 03 21:33:40.360371 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:40.360347    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:40.360547 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:40.360527    4993 patch_prober.go:29] interesting pod/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 03 21:33:40.360591 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:40.360573    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs" podUID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 03 21:33:41.360461 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:41.360432    4993 patch_prober.go:29] interesting pod/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 03 21:33:41.360885 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:41.360481    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs" podUID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 03 21:33:42.360937 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:42.360907    4993 patch_prober.go:29] interesting pod/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 03 21:33:42.361366 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:42.360956    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs" podUID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 03 21:33:43.360027 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:43.359995    4993 patch_prober.go:29] interesting pod/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 03 21:33:43.360186 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:43.360046    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs" podUID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 03 21:33:44.360606 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:44.360572    4993 patch_prober.go:29] interesting pod/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 03 21:33:44.361026 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:44.360621    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs" podUID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 03 21:33:45.360760 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:45.360723    4993 patch_prober.go:29] interesting pod/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused" start-of-body=
Nov 03 21:33:45.361158 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:45.360806    4993 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs" podUID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.90:15021/healthz/ready\": dial tcp 10.133.0.90:15021: connect: connection refused"
Nov 03 21:33:46.364829 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:46.364784    4993 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:46.365261 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:46.365240    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:33:46.365857 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:33:46.365837    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:34:01.669316 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:34:01.669279    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:34:31.669036 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:34:31.668997    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:35:01.669262 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:35:01.669221    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:35:15.515753 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:15.515711    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"]
Nov 03 21:35:15.630454 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:15.630423    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"]
Nov 03 21:35:20.517057 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.517016    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq" podUID="f20f2047-7442-4403-882e-1f51302d4546" containerName="payload-processing" containerID="cri-o://8dafc9f1c0b429841b4cb507abf79bd594c61971904c59b06c86a80eb7ca2de5" gracePeriod=25
Nov 03 21:35:20.630873 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.630834    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k" podUID="8d159865-b910-40dc-b8a8-8c765ffbaf5e" containerName="payload-pre-processing" containerID="cri-o://7b2ec461c02352e14c539607c1544d4f334f22eba43980b27625a27b8fd4395d" gracePeriod=25
Nov 03 21:35:20.775049 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.774983    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"
Nov 03 21:35:20.793787 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.793757    4993 generic.go:356] "Generic (PLEG): container finished" podID="8d159865-b910-40dc-b8a8-8c765ffbaf5e" containerID="7b2ec461c02352e14c539607c1544d4f334f22eba43980b27625a27b8fd4395d" exitCode=0
Nov 03 21:35:20.793952 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.793835    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k" event={"ID":"8d159865-b910-40dc-b8a8-8c765ffbaf5e","Type":"ContainerDied","Data":"7b2ec461c02352e14c539607c1544d4f334f22eba43980b27625a27b8fd4395d"}
Nov 03 21:35:20.795141 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.795116    4993 generic.go:356] "Generic (PLEG): container finished" podID="f20f2047-7442-4403-882e-1f51302d4546" containerID="8dafc9f1c0b429841b4cb507abf79bd594c61971904c59b06c86a80eb7ca2de5" exitCode=0
Nov 03 21:35:20.795254 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.795165    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"
Nov 03 21:35:20.795254 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.795208    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq" event={"ID":"f20f2047-7442-4403-882e-1f51302d4546","Type":"ContainerDied","Data":"8dafc9f1c0b429841b4cb507abf79bd594c61971904c59b06c86a80eb7ca2de5"}
Nov 03 21:35:20.795337 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.795261    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq" event={"ID":"f20f2047-7442-4403-882e-1f51302d4546","Type":"ContainerDied","Data":"f96d7e888950b6eea20e225476c89e02b9faca77268259a35112e4e52d08e4a3"}
Nov 03 21:35:20.795337 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.795289    4993 scope.go:122] "RemoveContainer" containerID="8dafc9f1c0b429841b4cb507abf79bd594c61971904c59b06c86a80eb7ca2de5"
Nov 03 21:35:20.805679 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.805656    4993 scope.go:122] "RemoveContainer" containerID="8dafc9f1c0b429841b4cb507abf79bd594c61971904c59b06c86a80eb7ca2de5"
Nov 03 21:35:20.806004 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:35:20.805975    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dafc9f1c0b429841b4cb507abf79bd594c61971904c59b06c86a80eb7ca2de5\": container with ID starting with 8dafc9f1c0b429841b4cb507abf79bd594c61971904c59b06c86a80eb7ca2de5 not found: ID does not exist" containerID="8dafc9f1c0b429841b4cb507abf79bd594c61971904c59b06c86a80eb7ca2de5"
Nov 03 21:35:20.806106 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.806006    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dafc9f1c0b429841b4cb507abf79bd594c61971904c59b06c86a80eb7ca2de5"} err="failed to get container status \"8dafc9f1c0b429841b4cb507abf79bd594c61971904c59b06c86a80eb7ca2de5\": rpc error: code = NotFound desc = could not find container \"8dafc9f1c0b429841b4cb507abf79bd594c61971904c59b06c86a80eb7ca2de5\": container with ID starting with 8dafc9f1c0b429841b4cb507abf79bd594c61971904c59b06c86a80eb7ca2de5 not found: ID does not exist"
Nov 03 21:35:20.866384 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.866357    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f20f2047-7442-4403-882e-1f51302d4546-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f20f2047-7442-4403-882e-1f51302d4546-plugins-config-volume\") pod \"f20f2047-7442-4403-882e-1f51302d4546\" (UID: \"f20f2047-7442-4403-882e-1f51302d4546\") "
Nov 03 21:35:20.866547 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.866412    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f20f2047-7442-4403-882e-1f51302d4546-kube-api-access-xvqq7\" (UniqueName: \"kubernetes.io/projected/f20f2047-7442-4403-882e-1f51302d4546-kube-api-access-xvqq7\") pod \"f20f2047-7442-4403-882e-1f51302d4546\" (UID: \"f20f2047-7442-4403-882e-1f51302d4546\") "
Nov 03 21:35:20.866749 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.866729    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f20f2047-7442-4403-882e-1f51302d4546-plugins-config-volume" pod "f20f2047-7442-4403-882e-1f51302d4546" (UID: "f20f2047-7442-4403-882e-1f51302d4546"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:35:20.868985 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.868966    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f20f2047-7442-4403-882e-1f51302d4546-kube-api-access-xvqq7" pod "f20f2047-7442-4403-882e-1f51302d4546" (UID: "f20f2047-7442-4403-882e-1f51302d4546"). InnerVolumeSpecName "kube-api-access-xvqq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:35:20.896725 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.896704    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"
Nov 03 21:35:20.968293 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.968262    4993 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f20f2047-7442-4403-882e-1f51302d4546-plugins-config-volume\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:20.968466 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:20.968297    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvqq7\" (UniqueName: \"kubernetes.io/projected/f20f2047-7442-4403-882e-1f51302d4546-kube-api-access-xvqq7\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:21.069098 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:21.069025    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d159865-b910-40dc-b8a8-8c765ffbaf5e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d159865-b910-40dc-b8a8-8c765ffbaf5e-plugins-config-volume\") pod \"8d159865-b910-40dc-b8a8-8c765ffbaf5e\" (UID: \"8d159865-b910-40dc-b8a8-8c765ffbaf5e\") "
Nov 03 21:35:21.069098 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:21.069057    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d159865-b910-40dc-b8a8-8c765ffbaf5e-kube-api-access-v259j\" (UniqueName: \"kubernetes.io/projected/8d159865-b910-40dc-b8a8-8c765ffbaf5e-kube-api-access-v259j\") pod \"8d159865-b910-40dc-b8a8-8c765ffbaf5e\" (UID: \"8d159865-b910-40dc-b8a8-8c765ffbaf5e\") "
Nov 03 21:35:21.069386 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:21.069363    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d159865-b910-40dc-b8a8-8c765ffbaf5e-plugins-config-volume" pod "8d159865-b910-40dc-b8a8-8c765ffbaf5e" (UID: "8d159865-b910-40dc-b8a8-8c765ffbaf5e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:35:21.071295 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:21.071269    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d159865-b910-40dc-b8a8-8c765ffbaf5e-kube-api-access-v259j" pod "8d159865-b910-40dc-b8a8-8c765ffbaf5e" (UID: "8d159865-b910-40dc-b8a8-8c765ffbaf5e"). InnerVolumeSpecName "kube-api-access-v259j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:35:21.170249 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:21.170223    4993 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d159865-b910-40dc-b8a8-8c765ffbaf5e-plugins-config-volume\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:21.170348 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:21.170255    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v259j\" (UniqueName: \"kubernetes.io/projected/8d159865-b910-40dc-b8a8-8c765ffbaf5e-kube-api-access-v259j\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:21.213013 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:21.212983    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"]
Nov 03 21:35:21.217471 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:21.217448    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-authcfg-http-220932-77dccdc5b-5cwxq"]
Nov 03 21:35:21.575036 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:21.575009    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f20f2047-7442-4403-882e-1f51302d4546" path="/var/lib/kubelet/pods/f20f2047-7442-4403-882e-1f51302d4546/volumes"
Nov 03 21:35:21.802261 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:21.802189    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k" event={"ID":"8d159865-b910-40dc-b8a8-8c765ffbaf5e","Type":"ContainerDied","Data":"bdea6e413fe37c72528912ce488b018f949dc4e50e909ca4ae9f9b311d146c83"}
Nov 03 21:35:21.802261 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:21.802241    4993 scope.go:122] "RemoveContainer" containerID="7b2ec461c02352e14c539607c1544d4f334f22eba43980b27625a27b8fd4395d"
Nov 03 21:35:21.802261 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:21.802244    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"
Nov 03 21:35:21.964440 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:21.964415    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"]
Nov 03 21:35:21.970248 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:21.970226    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-authcfg-http-220932-58c8dff875-mt27k"]
Nov 03 21:35:23.575408 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:23.575376    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d159865-b910-40dc-b8a8-8c765ffbaf5e" path="/var/lib/kubelet/pods/8d159865-b910-40dc-b8a8-8c765ffbaf5e/volumes"
Nov 03 21:35:24.036654 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.036596    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"]
Nov 03 21:35:24.036853 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.036826    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd" podUID="6a9e8850-b0b9-4904-86d1-274359e5af0f" containerName="maas-api" containerID="cri-o://d9a1fd84f45b6e749efd00a788fb576bf9b3e4aeab583fbe9bb0513da2efaa55" gracePeriod=30
Nov 03 21:35:24.287778 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.287723    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:35:24.294890 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.294862    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a9e8850-b0b9-4904-86d1-274359e5af0f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9e8850-b0b9-4904-86d1-274359e5af0f-openshift-service-ca\") pod \"6a9e8850-b0b9-4904-86d1-274359e5af0f\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") "
Nov 03 21:35:24.295018 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.294929    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-metrics-certs\") pod \"6a9e8850-b0b9-4904-86d1-274359e5af0f\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") "
Nov 03 21:35:24.295018 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.295002    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a9e8850-b0b9-4904-86d1-274359e5af0f-kube-api-access-chmjw\" (UniqueName: \"kubernetes.io/projected/6a9e8850-b0b9-4904-86d1-274359e5af0f-kube-api-access-chmjw\") pod \"6a9e8850-b0b9-4904-86d1-274359e5af0f\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") "
Nov 03 21:35:24.295129 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.295041    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-maas-api-tls\") pod \"6a9e8850-b0b9-4904-86d1-274359e5af0f\" (UID: \"6a9e8850-b0b9-4904-86d1-274359e5af0f\") "
Nov 03 21:35:24.295231 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.295205    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a9e8850-b0b9-4904-86d1-274359e5af0f-openshift-service-ca" pod "6a9e8850-b0b9-4904-86d1-274359e5af0f" (UID: "6a9e8850-b0b9-4904-86d1-274359e5af0f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:35:24.295334 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.295315    4993 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9e8850-b0b9-4904-86d1-274359e5af0f-openshift-service-ca\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:24.297219 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.297194    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-metrics-certs" pod "6a9e8850-b0b9-4904-86d1-274359e5af0f" (UID: "6a9e8850-b0b9-4904-86d1-274359e5af0f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:35:24.297412 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.297394    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a9e8850-b0b9-4904-86d1-274359e5af0f-kube-api-access-chmjw" pod "6a9e8850-b0b9-4904-86d1-274359e5af0f" (UID: "6a9e8850-b0b9-4904-86d1-274359e5af0f"). InnerVolumeSpecName "kube-api-access-chmjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:35:24.297947 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.297927    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-maas-api-tls" pod "6a9e8850-b0b9-4904-86d1-274359e5af0f" (UID: "6a9e8850-b0b9-4904-86d1-274359e5af0f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 03 21:35:24.396155 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.396131    4993 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-metrics-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:24.396155 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.396152    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chmjw\" (UniqueName: \"kubernetes.io/projected/6a9e8850-b0b9-4904-86d1-274359e5af0f-kube-api-access-chmjw\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:24.396284 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.396161    4993 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6a9e8850-b0b9-4904-86d1-274359e5af0f-maas-api-tls\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:24.818403 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.818368    4993 generic.go:356] "Generic (PLEG): container finished" podID="6a9e8850-b0b9-4904-86d1-274359e5af0f" containerID="d9a1fd84f45b6e749efd00a788fb576bf9b3e4aeab583fbe9bb0513da2efaa55" exitCode=0
Nov 03 21:35:24.818854 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.818413    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"
Nov 03 21:35:24.818854 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.818419    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd" event={"ID":"6a9e8850-b0b9-4904-86d1-274359e5af0f","Type":"ContainerDied","Data":"d9a1fd84f45b6e749efd00a788fb576bf9b3e4aeab583fbe9bb0513da2efaa55"}
Nov 03 21:35:24.818854 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.818471    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd" event={"ID":"6a9e8850-b0b9-4904-86d1-274359e5af0f","Type":"ContainerDied","Data":"32fee8486d7cd32df31aeec6a111011be099784ed3d044b70be3257ca2a2dc15"}
Nov 03 21:35:24.818854 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.818503    4993 scope.go:122] "RemoveContainer" containerID="d9a1fd84f45b6e749efd00a788fb576bf9b3e4aeab583fbe9bb0513da2efaa55"
Nov 03 21:35:24.827991 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.827971    4993 scope.go:122] "RemoveContainer" containerID="d9a1fd84f45b6e749efd00a788fb576bf9b3e4aeab583fbe9bb0513da2efaa55"
Nov 03 21:35:24.828301 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:35:24.828270    4993 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9a1fd84f45b6e749efd00a788fb576bf9b3e4aeab583fbe9bb0513da2efaa55\": container with ID starting with d9a1fd84f45b6e749efd00a788fb576bf9b3e4aeab583fbe9bb0513da2efaa55 not found: ID does not exist" containerID="d9a1fd84f45b6e749efd00a788fb576bf9b3e4aeab583fbe9bb0513da2efaa55"
Nov 03 21:35:24.828401 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.828307    4993 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9a1fd84f45b6e749efd00a788fb576bf9b3e4aeab583fbe9bb0513da2efaa55"} err="failed to get container status \"d9a1fd84f45b6e749efd00a788fb576bf9b3e4aeab583fbe9bb0513da2efaa55\": rpc error: code = NotFound desc = could not find container \"d9a1fd84f45b6e749efd00a788fb576bf9b3e4aeab583fbe9bb0513da2efaa55\": container with ID starting with d9a1fd84f45b6e749efd00a788fb576bf9b3e4aeab583fbe9bb0513da2efaa55 not found: ID does not exist"
Nov 03 21:35:24.981961 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:24.981926    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"]
Nov 03 21:35:25.030949 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:25.030921    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-authcfg-http-220932-689bb569f4-4dxqd"]
Nov 03 21:35:25.576172 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:25.576138    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a9e8850-b0b9-4904-86d1-274359e5af0f" path="/var/lib/kubelet/pods/6a9e8850-b0b9-4904-86d1-274359e5af0f/volumes"
Nov 03 21:35:31.669142 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:35:31.669107    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:35:32.693359 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:32.693327    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"]
Nov 03 21:35:32.693900 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:32.693528    4993 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs" podUID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" containerName="istio-proxy" containerID="cri-o://f27225b6aefd29dcd99fb753ee7f498be0a271b463e4bf93189d474c22ab4009" gracePeriod=30
Nov 03 21:35:37.878528 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:37.878499    4993 generic.go:356] "Generic (PLEG): container finished" podID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" containerID="f27225b6aefd29dcd99fb753ee7f498be0a271b463e4bf93189d474c22ab4009" exitCode=0
Nov 03 21:35:37.878984 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:37.878540    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs" event={"ID":"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02","Type":"ContainerDied","Data":"f27225b6aefd29dcd99fb753ee7f498be0a271b463e4bf93189d474c22ab4009"}
Nov 03 21:35:37.878984 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:37.878607    4993 scope.go:122] "RemoveContainer" containerID="64935c8a056d5ff2e185811bdbaf1094310400a8006a1856ae290198c74730c2"
Nov 03 21:35:37.936699 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:37.936678    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:35:38.008551 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.008530    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-data\") pod \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") "
Nov 03 21:35:38.008693 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.008582    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-kube-api-access-42x6k\" (UniqueName: \"kubernetes.io/projected/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-kube-api-access-42x6k\") pod \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") "
Nov 03 21:35:38.008693 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.008614    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-podinfo\") pod \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") "
Nov 03 21:35:38.008693 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.008685    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istiod-ca-cert\") pod \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") "
Nov 03 21:35:38.008851 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.008705    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-token\" (UniqueName: \"kubernetes.io/projected/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-token\") pod \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") "
Nov 03 21:35:38.008851 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.008752    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-workload-socket\") pod \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") "
Nov 03 21:35:38.008851 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.008778    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-credential-socket\") pod \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") "
Nov 03 21:35:38.008851 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.008827    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-workload-certs\") pod \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") "
Nov 03 21:35:38.009059 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.008845    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-data" pod "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" (UID: "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:35:38.009059 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.008853    4993 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-envoy\") pod \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\" (UID: \"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02\") "
Nov 03 21:35:38.009223 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.009127    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istiod-ca-cert" pod "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" (UID: "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 03 21:35:38.009223 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.009202    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-credential-socket" pod "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" (UID: "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:35:38.009355 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.009248    4993 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-data\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:38.009355 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.009268    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-workload-socket" pod "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" (UID: "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:35:38.009465 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.009394    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-workload-certs" pod "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" (UID: "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:35:38.010821 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.010802    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-kube-api-access-42x6k" pod "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" (UID: "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02"). InnerVolumeSpecName "kube-api-access-42x6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:35:38.010959 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.010940    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-token" pod "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" (UID: "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 03 21:35:38.011361 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.011344    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-envoy" pod "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" (UID: "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 03 21:35:38.011423 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.011356    4993 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-podinfo" pod "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" (UID: "ee92fab9-aa65-40ea-9c8f-7cd7f2142e02"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 03 21:35:38.110247 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.110225    4993 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istiod-ca-cert\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:38.110247 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.110247    4993 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-token\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:38.110247 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.110257    4993 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-workload-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:38.110432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.110267    4993 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-credential-socket\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:38.110432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.110277    4993 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-workload-certs\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:38.110432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.110286    4993 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-envoy\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:38.110432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.110294    4993 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42x6k\" (UniqueName: \"kubernetes.io/projected/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-kube-api-access-42x6k\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:38.110432 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.110303    4993 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02-istio-podinfo\") on node \"ip-10-0-142-223.us-west-1.compute.internal\" DevicePath \"\""
Nov 03 21:35:38.894088 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.894050    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs" event={"ID":"ee92fab9-aa65-40ea-9c8f-7cd7f2142e02","Type":"ContainerDied","Data":"c92869840a3dd0d79f91f96a466a1ec1dc315eb0bb14532c0526f49b7eea4afe"}
Nov 03 21:35:38.894568 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.894099    4993 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"
Nov 03 21:35:38.894568 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:38.894139    4993 scope.go:122] "RemoveContainer" containerID="f27225b6aefd29dcd99fb753ee7f498be0a271b463e4bf93189d474c22ab4009"
Nov 03 21:35:39.107976 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:39.107937    4993 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"]
Nov 03 21:35:39.157071 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:39.156987    4993 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-authcfg-http-220932-openshift-default-68ccb5c9c7-qskbs"]
Nov 03 21:35:39.582094 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:35:39.582056    4993 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" path="/var/lib/kubelet/pods/ee92fab9-aa65-40ea-9c8f-7cd7f2142e02/volumes"
Nov 03 21:36:01.668184 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:36:01.668147    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:36:31.669461 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:36:31.669424    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:37:01.669651 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:37:01.669591    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:37:04.802299 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:37:04.802221    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-658lq_b2744188-2043-4fd3-85c1-ed75eba9aae2/manager/0.log"
Nov 03 21:37:05.679818 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:37:05.679789    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-4fm4w_5b525af5-eb4e-4ca6-bca2-71eb924f1a30/server/0.log"
Nov 03 21:37:05.910622 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:37:05.910592    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-5vf4s_4f18b897-8328-469a-a7ec-89132fde8f95/manager/0.log"
Nov 03 21:37:07.253056 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:37:07.253026    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-76d76db597-cthjw_97a2836b-4d21-42ab-b233-fba1a3cc887c/maas-api/0.log"
Nov 03 21:37:08.205693 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:37:08.205667    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-85hv9_c3ad1a5d-d272-43d5-9e6c-0c18e37f6203/postgres/0.log"
Nov 03 21:37:10.152744 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:37:10.152704    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-zkjdp_5273aca2-0270-419a-8665-b6354bd7ea9a/manager/0.log"
Nov 03 21:37:13.955287 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:37:13.955260    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7_b056b057-0700-4450-b07d-f19b53d5584a/istio-proxy/0.log"
Nov 03 21:37:14.282888 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:37:14.282858    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-cws5p_6bf4e14e-9896-452a-bfe9-58e3f89a155c/istio-proxy/0.log"
Nov 03 21:37:14.547611 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:37:14.547535    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-g24jg_ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b/istio-proxy/0.log"
Nov 03 21:37:31.670287 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:37:31.670248    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:37:39.640515 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:37:39.640480    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-658lq_b2744188-2043-4fd3-85c1-ed75eba9aae2/manager/0.log"
Nov 03 21:37:50.488071 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:37:50.488033    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-4fm4w_5b525af5-eb4e-4ca6-bca2-71eb924f1a30/server/0.log"
Nov 03 21:37:50.720917 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:37:50.720884    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-5vf4s_4f18b897-8328-469a-a7ec-89132fde8f95/manager/0.log"
Nov 03 21:38:01.668875 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:38:01.668837    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:38:02.118896 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:38:02.118859    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-76d76db597-cthjw_97a2836b-4d21-42ab-b233-fba1a3cc887c/maas-api/0.log"
Nov 03 21:38:03.099182 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:38:03.099142    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-85hv9_c3ad1a5d-d272-43d5-9e6c-0c18e37f6203/postgres/0.log"
Nov 03 21:38:04.711367 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:38:04.711336    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-zkjdp_5273aca2-0270-419a-8665-b6354bd7ea9a/manager/0.log"
Nov 03 21:38:08.130668 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:38:08.130622    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-b2dl7_b056b057-0700-4450-b07d-f19b53d5584a/istio-proxy/0.log"
Nov 03 21:38:08.384268 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:38:08.384190    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-cws5p_6bf4e14e-9896-452a-bfe9-58e3f89a155c/istio-proxy/0.log"
Nov 03 21:38:08.660186 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:38:08.660102    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-g24jg_ad5c0f32-0285-4b4b-bf4a-db9ef6b12d1b/istio-proxy/0.log"
Nov 03 21:38:31.669362 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:38:31.669313    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:39:01.669430 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:39:01.669385    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:39:31.668914 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:39:31.668875    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:40:01.669351 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:40:01.669314    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:40:25.323727 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.323690    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fd9m6/must-gather-vdxvc"]
Nov 03 21:40:25.324184 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.324034    4993 cpu_manager.go:414] "containerMap: removing container" podUID="6a9e8850-b0b9-4904-86d1-274359e5af0f" containerName="maas-api"
Nov 03 21:40:25.324184 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.324045    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a9e8850-b0b9-4904-86d1-274359e5af0f" containerName="maas-api"
Nov 03 21:40:25.324184 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.324064    4993 cpu_manager.go:414] "containerMap: removing container" podUID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" containerName="istio-proxy"
Nov 03 21:40:25.324184 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.324069    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" containerName="istio-proxy"
Nov 03 21:40:25.324184 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.324082    4993 cpu_manager.go:414] "containerMap: removing container" podUID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" containerName="istio-proxy"
Nov 03 21:40:25.324184 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.324088    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" containerName="istio-proxy"
Nov 03 21:40:25.324184 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.324100    4993 cpu_manager.go:414] "containerMap: removing container" podUID="f20f2047-7442-4403-882e-1f51302d4546" containerName="payload-processing"
Nov 03 21:40:25.324184 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.324104    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f20f2047-7442-4403-882e-1f51302d4546" containerName="payload-processing"
Nov 03 21:40:25.324184 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.324114    4993 cpu_manager.go:414] "containerMap: removing container" podUID="8d159865-b910-40dc-b8a8-8c765ffbaf5e" containerName="payload-pre-processing"
Nov 03 21:40:25.324184 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.324119    4993 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d159865-b910-40dc-b8a8-8c765ffbaf5e" containerName="payload-pre-processing"
Nov 03 21:40:25.324184 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.324185    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="f20f2047-7442-4403-882e-1f51302d4546" containerName="payload-processing"
Nov 03 21:40:25.324543 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.324193    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d159865-b910-40dc-b8a8-8c765ffbaf5e" containerName="payload-pre-processing"
Nov 03 21:40:25.324543 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.324201    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a9e8850-b0b9-4904-86d1-274359e5af0f" containerName="maas-api"
Nov 03 21:40:25.324543 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.324208    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" containerName="istio-proxy"
Nov 03 21:40:25.324543 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.324214    4993 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee92fab9-aa65-40ea-9c8f-7cd7f2142e02" containerName="istio-proxy"
Nov 03 21:40:25.327296 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.327277    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fd9m6/must-gather-vdxvc"
Nov 03 21:40:25.379492 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.379452    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fd9m6\"/\"openshift-service-ca.crt\""
Nov 03 21:40:25.409363 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.409325    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbpc9\" (UniqueName: \"kubernetes.io/projected/9a0b3c62-fdf6-4a83-bc13-38a8d8fae551-kube-api-access-dbpc9\") pod \"must-gather-vdxvc\" (UID: \"9a0b3c62-fdf6-4a83-bc13-38a8d8fae551\") " pod="openshift-must-gather-fd9m6/must-gather-vdxvc"
Nov 03 21:40:25.409530 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.409396    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9a0b3c62-fdf6-4a83-bc13-38a8d8fae551-must-gather-output\") pod \"must-gather-vdxvc\" (UID: \"9a0b3c62-fdf6-4a83-bc13-38a8d8fae551\") " pod="openshift-must-gather-fd9m6/must-gather-vdxvc"
Nov 03 21:40:25.428692 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.428661    4993 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fd9m6\"/\"kube-root-ca.crt\""
Nov 03 21:40:25.428808 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.428717    4993 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fd9m6\"/\"default-dockercfg-9chns\""
Nov 03 21:40:25.430000 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.429980    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fd9m6/must-gather-vdxvc"]
Nov 03 21:40:25.510296 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.510271    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9a0b3c62-fdf6-4a83-bc13-38a8d8fae551-must-gather-output\") pod \"must-gather-vdxvc\" (UID: \"9a0b3c62-fdf6-4a83-bc13-38a8d8fae551\") " pod="openshift-must-gather-fd9m6/must-gather-vdxvc"
Nov 03 21:40:25.510406 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.510335    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbpc9\" (UniqueName: \"kubernetes.io/projected/9a0b3c62-fdf6-4a83-bc13-38a8d8fae551-kube-api-access-dbpc9\") pod \"must-gather-vdxvc\" (UID: \"9a0b3c62-fdf6-4a83-bc13-38a8d8fae551\") " pod="openshift-must-gather-fd9m6/must-gather-vdxvc"
Nov 03 21:40:25.510616 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.510598    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9a0b3c62-fdf6-4a83-bc13-38a8d8fae551-must-gather-output\") pod \"must-gather-vdxvc\" (UID: \"9a0b3c62-fdf6-4a83-bc13-38a8d8fae551\") " pod="openshift-must-gather-fd9m6/must-gather-vdxvc"
Nov 03 21:40:25.570606 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.570585    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbpc9\" (UniqueName: \"kubernetes.io/projected/9a0b3c62-fdf6-4a83-bc13-38a8d8fae551-kube-api-access-dbpc9\") pod \"must-gather-vdxvc\" (UID: \"9a0b3c62-fdf6-4a83-bc13-38a8d8fae551\") " pod="openshift-must-gather-fd9m6/must-gather-vdxvc"
Nov 03 21:40:25.657403 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:25.657350    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fd9m6/must-gather-vdxvc"
Nov 03 21:40:26.027380 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:26.027357    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fd9m6/must-gather-vdxvc"]
Nov 03 21:40:26.033454 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:40:26.033318    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a0b3c62_fdf6_4a83_bc13_38a8d8fae551.slice/crio-9c8b9877eb0e186eb01cd1e282be841ba04cec466bc6313286f6300b3572abd3 WatchSource:0}: Error finding container 9c8b9877eb0e186eb01cd1e282be841ba04cec466bc6313286f6300b3572abd3: Status 404 returned error can't find the container with id 9c8b9877eb0e186eb01cd1e282be841ba04cec466bc6313286f6300b3572abd3
Nov 03 21:40:26.034951 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:26.034933    4993 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 03 21:40:26.175759 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:26.175728    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fd9m6/must-gather-vdxvc" event={"ID":"9a0b3c62-fdf6-4a83-bc13-38a8d8fae551","Type":"ContainerStarted","Data":"9c8b9877eb0e186eb01cd1e282be841ba04cec466bc6313286f6300b3572abd3"}
Nov 03 21:40:28.187186 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:28.187149    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fd9m6/must-gather-vdxvc" event={"ID":"9a0b3c62-fdf6-4a83-bc13-38a8d8fae551","Type":"ContainerStarted","Data":"b68e3928dfc6e01fc58f122824e5515985b29051992050ffc9082d47d3d274bf"}
Nov 03 21:40:28.187726 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:28.187196    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fd9m6/must-gather-vdxvc" event={"ID":"9a0b3c62-fdf6-4a83-bc13-38a8d8fae551","Type":"ContainerStarted","Data":"09c17f04d1df55ea5f4686ee185be81b9b5a5a58a9723158d8acd931c1697557"}
Nov 03 21:40:31.673173 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:40:31.673126    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:40:34.110822 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:34.101127    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8jhcx_9353a680-00bf-4d3a-8437-5a959a40a682/global-pull-secret-syncer/0.log"
Nov 03 21:40:37.707469 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:37.707434    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-h952v_b6fe25b0-03dc-434d-957b-5f25c8d5c141/konnectivity-agent/0.log"
Nov 03 21:40:39.503520 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:39.503481    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-223.us-west-1.compute.internal_88d138da692bc0e2c453d9d2e3886bfc/haproxy/0.log"
Nov 03 21:40:43.785539 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:43.785491    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-zkjdp_5273aca2-0270-419a-8665-b6354bd7ea9a/manager/0.log"
Nov 03 21:40:44.150465 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.150352    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-fd9m6/must-gather-vdxvc" podStartSLOduration=17.959740967 podStartE2EDuration="19.150333903s" totalImagesPullingTime="1.190592936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-03 21:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:40:28.310929598 +0000 UTC m=+3630.855635974" watchObservedRunningTime="2026-11-03 21:40:44.150333903 +0000 UTC m=+3646.695040275"
Nov 03 21:40:44.150653 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.150508    4993 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"]
Nov 03 21:40:44.157374 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.157352    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.244518 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.244487    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"]
Nov 03 21:40:44.319523 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.319483    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a6fcba5c-e9e6-4d0d-803a-80126c543cea-lib-modules\") pod \"perf-node-gather-daemonset-thqn9\" (UID: \"a6fcba5c-e9e6-4d0d-803a-80126c543cea\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.319718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.319607    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4grg\" (UniqueName: \"kubernetes.io/projected/a6fcba5c-e9e6-4d0d-803a-80126c543cea-kube-api-access-r4grg\") pod \"perf-node-gather-daemonset-thqn9\" (UID: \"a6fcba5c-e9e6-4d0d-803a-80126c543cea\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.319718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.319683    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6fcba5c-e9e6-4d0d-803a-80126c543cea-sys\") pod \"perf-node-gather-daemonset-thqn9\" (UID: \"a6fcba5c-e9e6-4d0d-803a-80126c543cea\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.319718 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.319702    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a6fcba5c-e9e6-4d0d-803a-80126c543cea-podres\") pod \"perf-node-gather-daemonset-thqn9\" (UID: \"a6fcba5c-e9e6-4d0d-803a-80126c543cea\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.319831 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.319723    4993 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a6fcba5c-e9e6-4d0d-803a-80126c543cea-proc\") pod \"perf-node-gather-daemonset-thqn9\" (UID: \"a6fcba5c-e9e6-4d0d-803a-80126c543cea\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.421155 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.421063    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a6fcba5c-e9e6-4d0d-803a-80126c543cea-podres\") pod \"perf-node-gather-daemonset-thqn9\" (UID: \"a6fcba5c-e9e6-4d0d-803a-80126c543cea\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.421155 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.421109    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a6fcba5c-e9e6-4d0d-803a-80126c543cea-proc\") pod \"perf-node-gather-daemonset-thqn9\" (UID: \"a6fcba5c-e9e6-4d0d-803a-80126c543cea\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.421380 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.421169    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a6fcba5c-e9e6-4d0d-803a-80126c543cea-lib-modules\") pod \"perf-node-gather-daemonset-thqn9\" (UID: \"a6fcba5c-e9e6-4d0d-803a-80126c543cea\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.421380 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.421199    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a6fcba5c-e9e6-4d0d-803a-80126c543cea-podres\") pod \"perf-node-gather-daemonset-thqn9\" (UID: \"a6fcba5c-e9e6-4d0d-803a-80126c543cea\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.421380 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.421249    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4grg\" (UniqueName: \"kubernetes.io/projected/a6fcba5c-e9e6-4d0d-803a-80126c543cea-kube-api-access-r4grg\") pod \"perf-node-gather-daemonset-thqn9\" (UID: \"a6fcba5c-e9e6-4d0d-803a-80126c543cea\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.421380 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.421264    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a6fcba5c-e9e6-4d0d-803a-80126c543cea-proc\") pod \"perf-node-gather-daemonset-thqn9\" (UID: \"a6fcba5c-e9e6-4d0d-803a-80126c543cea\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.421380 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.421272    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a6fcba5c-e9e6-4d0d-803a-80126c543cea-lib-modules\") pod \"perf-node-gather-daemonset-thqn9\" (UID: \"a6fcba5c-e9e6-4d0d-803a-80126c543cea\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.421380 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.421291    4993 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6fcba5c-e9e6-4d0d-803a-80126c543cea-sys\") pod \"perf-node-gather-daemonset-thqn9\" (UID: \"a6fcba5c-e9e6-4d0d-803a-80126c543cea\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.421380 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.421373    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a6fcba5c-e9e6-4d0d-803a-80126c543cea-sys\") pod \"perf-node-gather-daemonset-thqn9\" (UID: \"a6fcba5c-e9e6-4d0d-803a-80126c543cea\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.482136 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.482103    4993 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4grg\" (UniqueName: \"kubernetes.io/projected/a6fcba5c-e9e6-4d0d-803a-80126c543cea-kube-api-access-r4grg\") pod \"perf-node-gather-daemonset-thqn9\" (UID: \"a6fcba5c-e9e6-4d0d-803a-80126c543cea\") " pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:44.771662 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:44.771609    4993 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:45.147417 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:45.147331    4993 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"]
Nov 03 21:40:45.150008 ip-10-0-142-223 kubenswrapper[4993]: W1103 21:40:45.149981    4993 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda6fcba5c_e9e6_4d0d_803a_80126c543cea.slice/crio-a6e0fe154aaeefae351fa1f4a731246c10c91fedb505feaf1df721a1a884ce75 WatchSource:0}: Error finding container a6e0fe154aaeefae351fa1f4a731246c10c91fedb505feaf1df721a1a884ce75: Status 404 returned error can't find the container with id a6e0fe154aaeefae351fa1f4a731246c10c91fedb505feaf1df721a1a884ce75
Nov 03 21:40:45.318225 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:45.318197    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9" event={"ID":"a6fcba5c-e9e6-4d0d-803a-80126c543cea","Type":"ContainerStarted","Data":"a7b6a17658f7b9f564e4fa8316006b1622ef9e08506370b81453073589528665"}
Nov 03 21:40:45.318355 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:45.318254    4993 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9" event={"ID":"a6fcba5c-e9e6-4d0d-803a-80126c543cea","Type":"ContainerStarted","Data":"a6e0fe154aaeefae351fa1f4a731246c10c91fedb505feaf1df721a1a884ce75"}
Nov 03 21:40:45.318355 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:45.318280    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:45.435583 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:45.435442    4993 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9" podStartSLOduration=1.435424452 podStartE2EDuration="1.435424452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-03 21:40:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-03 21:40:45.433572557 +0000 UTC m=+3647.978278920" watchObservedRunningTime="2026-11-03 21:40:45.435424452 +0000 UTC m=+3647.980130823"
Nov 03 21:40:48.532950 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:48.532915    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14913c4a-3449-4529-81fd-eb1a64fbaa95/alertmanager/0.log"
Nov 03 21:40:48.946854 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:48.946781    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14913c4a-3449-4529-81fd-eb1a64fbaa95/config-reloader/0.log"
Nov 03 21:40:49.360110 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:49.360078    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14913c4a-3449-4529-81fd-eb1a64fbaa95/kube-rbac-proxy-web/0.log"
Nov 03 21:40:49.774198 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:49.774167    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14913c4a-3449-4529-81fd-eb1a64fbaa95/kube-rbac-proxy/0.log"
Nov 03 21:40:50.187574 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:50.187504    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14913c4a-3449-4529-81fd-eb1a64fbaa95/kube-rbac-proxy-metric/0.log"
Nov 03 21:40:50.601813 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:50.601778    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14913c4a-3449-4529-81fd-eb1a64fbaa95/prom-label-proxy/0.log"
Nov 03 21:40:51.016946 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:51.016890    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_14913c4a-3449-4529-81fd-eb1a64fbaa95/init-config-reloader/0.log"
Nov 03 21:40:51.335665 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:51.335566    4993 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fd9m6/perf-node-gather-daemonset-thqn9"
Nov 03 21:40:55.305953 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:55.305908    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6sjbh_737d534b-bee4-44e9-ac5d-124b25de7f72/node-exporter/0.log"
Nov 03 21:40:55.717462 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:55.717434    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6sjbh_737d534b-bee4-44e9-ac5d-124b25de7f72/kube-rbac-proxy/0.log"
Nov 03 21:40:56.144648 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:40:56.144547    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6sjbh_737d534b-bee4-44e9-ac5d-124b25de7f72/init-textfile/0.log"
Nov 03 21:41:01.670028 ip-10-0-142-223 kubenswrapper[4993]: E1103 21:41:01.669979    4993 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-j86dg" containerName="extractor"
Nov 03 21:41:02.079159 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:41:02.079126    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-8gqtt_1b5fff49-fcc5-419f-978f-afb7ea92e6c3/prometheus-operator/0.log"
Nov 03 21:41:02.489805 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:41:02.489763    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-8gqtt_1b5fff49-fcc5-419f-978f-afb7ea92e6c3/kube-rbac-proxy/0.log"
Nov 03 21:41:02.905475 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:41:02.905397    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-gwzjt_c4cacc5f-ade7-47d9-af40-78cdc6e090ba/prometheus-operator-admission-webhook/0.log"
Nov 03 21:41:04.583327 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:41:04.583302    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d45496948-n4hjf_9ba3faf5-eacc-43e1-9d00-78adbe1ecd00/thanos-query/0.log"
Nov 03 21:41:04.998826 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:41:04.998792    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d45496948-n4hjf_9ba3faf5-eacc-43e1-9d00-78adbe1ecd00/kube-rbac-proxy-web/0.log"
Nov 03 21:41:05.418052 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:41:05.417967    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d45496948-n4hjf_9ba3faf5-eacc-43e1-9d00-78adbe1ecd00/kube-rbac-proxy/0.log"
Nov 03 21:41:05.830920 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:41:05.830894    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d45496948-n4hjf_9ba3faf5-eacc-43e1-9d00-78adbe1ecd00/prom-label-proxy/0.log"
Nov 03 21:41:06.245697 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:41:06.245660    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d45496948-n4hjf_9ba3faf5-eacc-43e1-9d00-78adbe1ecd00/kube-rbac-proxy-rules/0.log"
Nov 03 21:41:06.659274 ip-10-0-142-223 kubenswrapper[4993]: I1103 21:41:06.659179    4993 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-d45496948-n4hjf_9ba3faf5-eacc-43e1-9d00-78adbe1ecd00/kube-rbac-proxy-metrics/0.log"